[{"data":1,"prerenderedAt":44571},["ShallowReactive",2],{"blog-post-/blogs/zsh":3,"all-blogs-for-navigation":400},{"id":4,"title":5,"body":6,"description":5,"extension":388,"meta":389,"navigation":325,"ogImage":391,"path":396,"seo":397,"stem":398,"__hash__":399},"content/blogs/3. 记录一下我的 zsh 配置.md","记录一下我的 zsh 配置",{"type":7,"value":8,"toc":371},"minimark",[9,13,17,21,24,27,30,33,36,92,95,99,102,105,178,181,200,203,206,210,213,216,236,239,242,244,263,265,272,275,278,281,291,294,297,300,303,306,361,364,367],[10,11,12],"h2",{"id":12},"背景",[14,15,16],"p",{},"最近想写点东西记录一下自己折腾这些工具的经历，之前没想到记录什么，今年实验室的服务器迁移需要重新配置一下 zsh,于是想着干脆顺手写一下自己折腾 zsh 出来的方案。",[10,18,20],{"id":19},"什么是-zsh","什么是 Zsh",[14,22,23],{},"Zsh 也是一种 shell 软件，可以看成是一个更好用的 Bash，目前 MacOS 中的默认 shell 已经是 Zsh 了。Zsh 完全兼容 Bash，拥有极其丰富的插件，其强大的自动补全参数、文件名以及自定义功能，可以大大提高我们使用 Linux 的效率。",[14,25,26],{},"Zsh 可以通过大部分主流 Linux 发行版的包管理器进行安装，若在没有 sudo 权限的服务器上也可以通过conda进行安装。",[10,28,29],{"id":29},"oh-my-zsh",[14,31,32],{},"Zsh 固然好用，但初始时只有白板没有任何插件，距离理想的使用体验还有一些差距，这里我们介绍 oh-my-zsh。",[14,34,35],{},"oh-my-zsh 是社区为 zsh 收集的一个第三方插件和主题合集，我们可以通过国内镜像快速安装，这里使用了南京大学镜像站。",[37,38,43],"pre",{"className":39,"code":40,"language":41,"meta":42,"style":42},"language-bash shiki shiki-themes github-light dracula","git clone https://mirror.nju.edu.cn/git/ohmyzsh.git\ncd ohmyzsh/tools\nREMOTE=https://mirror.nju.edu.cn/git/ohmyzsh.git sh install.sh\n","bash","",[44,45,46,62,72],"code",{"__ignoreMap":42},[47,48,51,55,59],"span",{"class":49,"line":50},"line",1,[47,52,54],{"class":53},"sCLZk","git",[47,56,58],{"class":57},"sEzAm"," clone",[47,60,61],{"class":57}," https://mirror.nju.edu.cn/git/ohmyzsh.git\n",[47,63,65,69],{"class":49,"line":64},2,[47,66,68],{"class":67},"sDgm9","cd",[47,70,71],{"class":57}," ohmyzsh/tools\n",[47,73,75,79,83,86,89],{"class":49,"line":74},3,[47,76,78],{"class":77},"scEb-","REMOTE",[47,80,82],{"class":81},"szJfE","=",[47,84,85],{"class":57},"https://mirror.nju.edu.cn/git/ohmyzsh.git",[47,87,88],{"class":53}," sh",[47,90,91],{"class":57}," install.sh\n",[14,93,94],{},"通过这三条指令可以在境内快速安装我们的 oh-my-zsh，同时也可以选择将 zsh 设置为默认 shell。",[10,96,98],{"id":97},"powerlevel10k","Powerlevel10k",[14,100,101],{},"oh-my-zsh 中的主题我认为还是不够美观和定制，我希望有个更好的体验，这里我们引入 Powerlevel10k 主题。在众多主题中，Powerlevel10k 因其美观性和高度定制的特性脱颖而出，截止到 2026年4月28日，Powerlevel10k 在 github 已累计收获 54k 个 star。",[14,103,104],{},"在国内，安装 Powerlevel10k 可以使用 gitee 的镜像进行安装，安装指令如下：",[37,106,108],{"className":39,"code":107,"language":41,"meta":42,"style":42},"git clone --depth=1 https://gitee.com/romkatv/powerlevel10k.git \"${ZSH_CUSTOM:-$HOME/.oh-my-zsh/custom}/themes/powerlevel10k\"\necho 'source /usr/share/zsh-theme-powerlevel10k/powerlevel10k.zsh-theme' >>~/.zshrc\n",[44,109,110,158],{"__ignoreMap":42},[47,111,112,114,116,120,123,127,130,133,136,139,142,145,147,149,152,155],{"class":49,"line":50},[47,113,54],{"class":53},[47,115,58],{"class":57},[47,117,119],{"class":118},"soDru"," --depth=1",[47,121,122],{"class":57}," https://gitee.com/romkatv/powerlevel10k.git",[47,124,126],{"class":125},"sMWOi"," \"",[47,128,129],{"class":57},"${",[47,131,132],{"class":77},"ZSH_CUSTOM",[47,134,135],{"class":81},":-",[47,137,138],{"class":77},"$HOME",[47,140,141],{"class":81},"/",[47,143,144],{"class":57},".",[47,146,29],{"class":77},[47,148,141],{"class":81},[47,150,151],{"class":77},"custom",[47,153,154],{"class":57},"}/themes/powerlevel10k",[47,156,157],{"class":125},"\"\n",[47,159,160,163,166,169,172,175],{"class":49,"line":64},[47,161,162],{"class":67},"echo",[47,164,165],{"class":125}," '",[47,167,168],{"class":57},"source /usr/share/zsh-theme-powerlevel10k/powerlevel10k.zsh-theme",[47,170,171],{"class":125},"'",[47,173,174],{"class":81}," >>",[47,176,177],{"class":57},"~/.zshrc\n",[14,179,180],{},"之后需要安装一款推荐字体，这里我个人使用的 0xProtoNerdFont 字体，大家也可以在其他博客或者官方仓库中使用其他人推荐的字体。",[14,182,183,184,187,188,191,192,195,196,199],{},"之后修改 ",[44,185,186],{},"~/.zshrc"," 文件，将变量 ",[44,189,190],{},"ZSH_THEME"," 修改为 ",[44,193,194],{},"powerlevel10k/powerlevel10k",",即 ",[44,197,198],{},"ZSH_THEME=\"powerlevel10k/powerlevel10k\""," 即可，之后重启终端即可根据个人喜好开始配置。",[10,201,202],{"id":202},"插件设置",[14,204,205],{},"现在主题已经搞定了，下一步开始设置真正能帮助我们提高生产力的插件。",[207,208,209],"h3",{"id":209},"zsh-autosuggestions",[14,211,212],{},"一个非常有用的插件，它根据历史记录和补全信息在输入时建议命令。",[14,214,215],{},"安装指令：",[37,217,219],{"className":39,"code":218,"language":41,"meta":42,"style":42},"git clone https://github.com/zsh-users/zsh-autosuggestions $ZSH_CUSTOM/plugins/zsh-autosuggestions\n",[44,220,221],{"__ignoreMap":42},[47,222,223,225,227,230,233],{"class":49,"line":50},[47,224,54],{"class":53},[47,226,58],{"class":57},[47,228,229],{"class":57}," https://github.com/zsh-users/zsh-autosuggestions",[47,231,232],{"class":77}," $ZSH_CUSTOM",[47,234,235],{"class":57},"/plugins/zsh-autosuggestions\n",[207,237,238],{"id":238},"zsh-syntax-highlighting",[14,240,241],{},"提供指令和文件路径的高亮限制，方便你检查指令和路径的正确性。",[14,243,215],{},[37,245,247],{"className":39,"code":246,"language":41,"meta":42,"style":42},"git clone https://github.com/zsh-users/zsh-syntax-highlighting $ZSH_CUSTOM/plugins/zsh-syntax-highlighting\n",[44,248,249],{"__ignoreMap":42},[47,250,251,253,255,258,260],{"class":49,"line":50},[47,252,54],{"class":53},[47,254,58],{"class":57},[47,256,257],{"class":57}," https://github.com/zsh-users/zsh-syntax-highlighting",[47,259,232],{"class":77},[47,261,262],{"class":57},"/plugins/zsh-syntax-highlighting\n",[207,264,54],{"id":54},[14,266,267,268,271],{},"oh-my-zsh 自带的插件，将大量常用的 git 指令简化，具体怎么使用可以参考 ",[44,269,270],{},"$ZSH/plugins/git/git.plugin.zsh "," 这个文件。",[10,273,274],{"id":274},"一些软件",[207,276,277],{"id":277},"nvm",[14,279,280],{},"一个好用的 node.js 版本管理器，无需 sudo 权限。",[14,282,283,284,290],{},"参考 ",[285,286,287],"a",{"href":287,"rel":288},"https://github.com/nvm-sh/nvm",[289],"nofollow"," 安装。",[207,292,293],{"id":293},"conda",[14,295,296],{},"无需多言。",[207,298,299],{"id":299},"fastfetch",[14,301,302],{},"获取本机发行版和硬件信息，比较酷炫，装逼使用。",[14,304,305],{},"可以进行美化，参考指令：",[37,307,309],{"className":39,"code":308,"language":41,"meta":42,"style":42},"cd $HOME/.config\n\n# 下载并解压然后删除\nwget https://github.com/sqlsec/fastfetch/releases/download/v0.1/fastfetch-for-Linux.zip\nunzip fastfetch-for-Linux.zip && rm fastfetch-for-Linux.zip\n",[44,310,311,321,327,333,342],{"__ignoreMap":42},[47,312,313,315,318],{"class":49,"line":50},[47,314,68],{"class":67},[47,316,317],{"class":77}," $HOME",[47,319,320],{"class":57},"/.config\n",[47,322,323],{"class":49,"line":64},[47,324,326],{"emptyLinePlaceholder":325},true,"\n",[47,328,329],{"class":49,"line":74},[47,330,332],{"class":331},"sfgPZ","# 下载并解压然后删除\n",[47,334,336,339],{"class":49,"line":335},4,[47,337,338],{"class":53},"wget",[47,340,341],{"class":57}," https://github.com/sqlsec/fastfetch/releases/download/v0.1/fastfetch-for-Linux.zip\n",[47,343,345,348,351,355,358],{"class":49,"line":344},5,[47,346,347],{"class":53},"unzip",[47,349,350],{"class":57}," fastfetch-for-Linux.zip",[47,352,354],{"class":353},"scbbO"," && ",[47,356,357],{"class":53},"rm",[47,359,360],{"class":57}," fastfetch-for-Linux.zip\n",[207,362,363],{"id":363},"onefetch",[14,365,366],{},"获取当前 git 项目信息，使用时需保证当前目录已经 git 初始化。",[368,369,370],"style",{},"html pre.shiki code .sCLZk, html code.shiki .sCLZk{--shiki-default:#6F42C1;--shiki-dark:#50FA7B}html pre.shiki code .sEzAm, html code.shiki .sEzAm{--shiki-default:#032F62;--shiki-dark:#F1FA8C}html pre.shiki code .sDgm9, html code.shiki .sDgm9{--shiki-default:#005CC5;--shiki-dark:#8BE9FD}html pre.shiki code .scEb-, html code.shiki .scEb-{--shiki-default:#24292E;--shiki-dark:#BD93F9}html pre.shiki code .szJfE, html code.shiki .szJfE{--shiki-default:#D73A49;--shiki-dark:#FF79C6}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html pre.shiki code .soDru, html code.shiki .soDru{--shiki-default:#005CC5;--shiki-dark:#BD93F9}html pre.shiki code .sMWOi, html code.shiki .sMWOi{--shiki-default:#032F62;--shiki-dark:#E9F284}html pre.shiki code .sfgPZ, html code.shiki .sfgPZ{--shiki-default:#6A737D;--shiki-dark:#6272A4}html pre.shiki code .scbbO, html code.shiki .scbbO{--shiki-default:#24292E;--shiki-dark:#F8F8F2}",{"title":42,"searchDepth":64,"depth":64,"links":372},[373,374,375,376,377,382],{"id":12,"depth":64,"text":12},{"id":19,"depth":64,"text":20},{"id":29,"depth":64,"text":29},{"id":97,"depth":64,"text":98},{"id":202,"depth":64,"text":202,"children":378},[379,380,381],{"id":209,"depth":74,"text":209},{"id":238,"depth":74,"text":238},{"id":54,"depth":74,"text":54},{"id":274,"depth":64,"text":274,"children":383},[384,385,386,387],{"id":277,"depth":74,"text":277},{"id":293,"depth":74,"text":293},{"id":299,"depth":74,"text":299},{"id":363,"depth":74,"text":363},"md",{"date":390,"image":391,"alt":392,"tags":393,"published":325},"28 April 2026","/blogs-img/blog3.jpg","PPO学习笔记",[394,395],"Linux","折腾笔记","/blogs/zsh",{"title":5,"description":5},"blogs/3. 记录一下我的 zsh 配置","y9ctoXQRoVK6l-0b4XyZbWKIXs5GhZ_Pvm3uVi1sgLU",[401,13028,44311],{"id":402,"title":403,"body":404,"description":13017,"extension":388,"meta":13018,"navigation":325,"ogImage":13020,"path":13024,"seo":13025,"stem":13026,"__hash__":13027},"content/blogs/1. transformer复习笔记.md","transformer 复习笔记",{"type":7,"value":405,"toc":13001},[406,408,411,426,429,436,439,442,445,704,707,831,1147,1577,1583,1854,2773,2776,2991,2994,3231,3238,3241,3244,3249,3581,4928,4945,5171,5630,5633,5728,5731,6318,6679,7035,7392,7665,7668,7671,8004,8161,8164,8167,8359,8498,8501,9013,9412,9416,9419,10028,10032,10035,11035,11038,11041,11331,11334,11491,11502,12095,12098,12804,12807,12985,12988,12993,12998],[10,407,12],{"id":12},[14,409,410],{},"距离首次学习 transformer 已经过去一年，最近在回忆 transformer 的相关内容，发现相关内容忘的差不多了，于是决定复习一下并记录下来。",[14,412,413,414,419,420,425],{},"本文主要记录对transformer的理解和思考，代码实现完全参考 ",[285,415,418],{"href":416,"rel":417},"https://zh.d2l.ai/chapter_attention-mechanisms/transformer.html",[289],"动手学深度学习"," 部分，部分理论的解释参考 ",[285,421,424],{"href":422,"rel":423},"https://nlp.seas.harvard.edu/annotated-transformer/",[289],"The Annotated Transformer"," 内容。",[10,427,428],{"id":428},"整体框架",[14,430,431],{},[432,433],"img",{"alt":434,"src":435},"transformer架构图","transformer/transformer.png",[10,437,438],{"id":438},"注意力机制",[207,440,441],{"id":441},"注意力函数",[14,443,444],{},"有两种常见的注意力函数，分别为加性注意力和缩放点积注意力。",[14,446,447,448,703],{},"点积注意力函数与缩放点积注意力函数基本相同，但缩放点积注意力函数有一个额外的缩放因子 ",[47,449,452,491],{"className":450},[451],"katex",[47,453,456],{"className":454},[455],"katex-mathml",[457,458,460],"math",{"xmlns":459},"http://www.w3.org/1998/Math/MathML",[461,462,463,486],"semantics",{},[464,465,466],"mrow",{},[467,468,469,473],"mfrac",{},[470,471,472],"mn",{},"1",[474,475,476],"msqrt",{},[477,478,479,483],"msub",{},[480,481,482],"mi",{},"d",[480,484,485],{},"k",[487,488,490],"annotation",{"encoding":489},"application/x-tex","\\frac{1}{\\sqrt{d_k}}",[47,492,496],{"className":493,"ariaHidden":495},[494],"katex-html","true",[47,497,500,505],{"className":498},[499],"base",[47,501],{"className":502,"style":504},[503],"strut","height:1.3831em;vertical-align:-0.538em;",[47,506,509,514,699],{"className":507},[508],"mord",[47,510],{"className":511},[512,513],"mopen","nulldelimiter",[47,515,517],{"className":516},[467],[47,518,522,690],{"className":519},[520,521],"vlist-t","vlist-t2",[47,523,526,687],{"className":524},[525],"vlist-r",[47,527,531,661,672],{"className":528,"style":530},[529],"vlist","height:0.8451em;",[47,532,534,539],{"style":533},"top:-2.5864em;",[47,535],{"className":536,"style":538},[537],"pstrut","height:3em;",[47,540,546],{"className":541},[542,543,544,545],"sizing","reset-size6","size3","mtight",[47,547,549],{"className":548},[508,545],[47,550,553],{"className":551},[508,552,545],"sqrt",[47,554,556,652],{"className":555},[520,521],[47,557,559,649],{"className":558},[525],[47,560,563,626],{"className":561,"style":562},[529],"height:0.8622em;",[47,564,568,571],{"className":565,"style":567},[566],"svg-align","top:-3em;",[47,569],{"className":570,"style":538},[537],[47,572,575],{"className":573,"style":574},[508,545],"padding-left:0.833em;",[47,576,578,582],{"className":577},[508,545],[47,579,482],{"className":580},[508,581,545],"mathnormal",[47,583,586],{"className":584},[585],"msupsub",[47,587,589,617],{"className":588},[520,521],[47,590,592,612],{"className":591},[525],[47,593,596],{"className":594,"style":595},[529],"height:0.3448em;",[47,597,599,603],{"style":598},"top:-2.3488em;margin-left:0em;margin-right:0.0714em;",[47,600],{"className":601,"style":602},[537],"height:2.5em;",[47,604,608],{"className":605},[542,606,607,545],"reset-size3","size1",[47,609,485],{"className":610,"style":611},[508,581,545],"margin-right:0.03148em;",[47,613,616],{"className":614},[615],"vlist-s","​",[47,618,620],{"className":619},[525],[47,621,624],{"className":622,"style":623},[529],"height:0.1512em;",[47,625],{},[47,627,629,632],{"style":628},"top:-2.8222em;",[47,630],{"className":631,"style":538},[537],[47,633,637],{"className":634,"style":636},[635,545],"hide-tail","min-width:0.853em;height:1.08em;",[638,639,645],"svg",{"xmlns":640,"width":641,"height":642,"viewBox":643,"preserveAspectRatio":644},"http://www.w3.org/2000/svg","400em","1.08em","0 0 400000 1080","xMinYMin slice",[646,647],"path",{"d":648},"M95,702\nc-2.7,0,-7.17,-2.7,-13.5,-8c-5.8,-5.3,-9.5,-10,-9.5,-14\nc0,-2,0.3,-3.3,1,-4c1.3,-2.7,23.83,-20.7,67.5,-54\nc44.2,-33.3,65.8,-50.3,66.5,-51c1.3,-1.3,3,-2,5,-2c4.7,0,8.7,3.3,12,10\ns173,378,173,378c0.7,0,35.3,-71,104,-213c68.7,-142,137.5,-285,206.5,-429\nc69,-144,104.5,-217.7,106.5,-221\nl0 -0\nc5.3,-9.3,12,-14,20,-14\nH400000v40H845.2724\ns-225.272,467,-225.272,467s-235,486,-235,486c-2.7,4.7,-9,7,-19,7\nc-6,0,-10,-1,-12,-3s-194,-422,-194,-422s-65,47,-65,47z\nM834 80h400000v40h-400000z",[47,650,616],{"className":651},[615],[47,653,655],{"className":654},[525],[47,656,659],{"className":657,"style":658},[529],"height:0.1778em;",[47,660],{},[47,662,664,667],{"style":663},"top:-3.23em;",[47,665],{"className":666,"style":538},[537],[47,668],{"className":669,"style":671},[670],"frac-line","border-bottom-width:0.04em;",[47,673,675,678],{"style":674},"top:-3.394em;",[47,676],{"className":677,"style":538},[537],[47,679,681],{"className":680},[542,543,544,545],[47,682,684],{"className":683},[508,545],[47,685,472],{"className":686},[508,545],[47,688,616],{"className":689},[615],[47,691,693],{"className":692},[525],[47,694,697],{"className":695,"style":696},[529],"height:0.538em;",[47,698],{},[47,700],{"className":701},[702,513],"mclose","。",[14,705,706],{},"加性注意力通过一个具有单个隐藏层的前馈网络计算兼容性函数。虽然两者在理论复杂度上相似，但在实际应用中，点积注意力更快且更节省空间，因为它可以使用高度优化的矩阵乘法代码实现。",[14,708,709,710,740,741,770,771,800,801,703],{},"在实践中，我们通常从小批量的角度来考虑提高效率，例如基于 ",[47,711,713,727],{"className":712},[451],[47,714,716],{"className":715},[455],[457,717,718],{"xmlns":459},[461,719,720,725],{},[464,721,722],{},[480,723,724],{},"n",[487,726,724],{"encoding":489},[47,728,730],{"className":729,"ariaHidden":495},[494],[47,731,733,737],{"className":732},[499],[47,734],{"className":735,"style":736},[503],"height:0.4306em;",[47,738,724],{"className":739},[508,581]," 个查询和 ",[47,742,744,758],{"className":743},[451],[47,745,747],{"className":746},[455],[457,748,749],{"xmlns":459},[461,750,751,756],{},[464,752,753],{},[480,754,755],{},"m",[487,757,755],{"encoding":489},[47,759,761],{"className":760,"ariaHidden":495},[494],[47,762,764,767],{"className":763},[499],[47,765],{"className":766,"style":736},[503],[47,768,755],{"className":769},[508,581]," 个键-值对计算注意力，其中查询和键的长度为 ",[47,772,774,787],{"className":773},[451],[47,775,777],{"className":776},[455],[457,778,779],{"xmlns":459},[461,780,781,785],{},[464,782,783],{},[480,784,482],{},[487,786,482],{"encoding":489},[47,788,790],{"className":789,"ariaHidden":495},[494],[47,791,793,797],{"className":792},[499],[47,794],{"className":795,"style":796},[503],"height:0.6944em;",[47,798,482],{"className":799},[508,581],"，值的长度为 ",[47,802,804,818],{"className":803},[451],[47,805,807],{"className":806},[455],[457,808,809],{"xmlns":459},[461,810,811,816],{},[464,812,813],{},[480,814,815],{},"v",[487,817,815],{"encoding":489},[47,819,821],{"className":820,"ariaHidden":495},[494],[47,822,824,827],{"className":823},[499],[47,825],{"className":826,"style":736},[503],[47,828,815],{"className":829,"style":830},[508,581],"margin-right:0.03588em;",[14,832,833,834,946,947,1046,1047,1146],{},"查询 ",[47,835,837,872],{"className":836},[451],[47,838,840],{"className":839},[455],[457,841,842],{"xmlns":459},[461,843,844,869],{},[464,845,846,849,853],{},[480,847,848],{},"Q",[850,851,852],"mo",{},"∈",[854,855,856,860],"msup",{},[480,857,859],{"mathvariant":858},"double-struck","R",[464,861,862,864,867],{},[480,863,724],{},[850,865,866],{},"×",[480,868,482],{},[487,870,871],{"encoding":489},"Q \\in \\mathbb{R}^{n \\times d}",[47,873,875,897],{"className":874,"ariaHidden":495},[494],[47,876,878,882,885,890,894],{"className":877},[499],[47,879],{"className":880,"style":881},[503],"height:0.8778em;vertical-align:-0.1944em;",[47,883,848],{"className":884},[508,581],[47,886],{"className":887,"style":889},[888],"mspace","margin-right:0.2778em;",[47,891,852],{"className":892},[893],"mrel",[47,895],{"className":896,"style":889},[888],[47,898,900,904],{"className":899},[499],[47,901],{"className":902,"style":903},[503],"height:0.8491em;",[47,905,907,911],{"className":906},[508],[47,908,859],{"className":909},[508,910],"mathbb",[47,912,914],{"className":913},[585],[47,915,917],{"className":916},[520],[47,918,920],{"className":919},[525],[47,921,923],{"className":922,"style":903},[529],[47,924,926,930],{"style":925},"top:-3.063em;margin-right:0.05em;",[47,927],{"className":928,"style":929},[537],"height:2.7em;",[47,931,933],{"className":932},[542,543,544,545],[47,934,936,939,943],{"className":935},[508,545],[47,937,724],{"className":938},[508,581,545],[47,940,866],{"className":941},[942,545],"mbin",[47,944,482],{"className":945},[508,581,545],"、键 ",[47,948,950,979],{"className":949},[451],[47,951,953],{"className":952},[455],[457,954,955],{"xmlns":459},[461,956,957,976],{},[464,958,959,962,964],{},[480,960,961],{},"K",[850,963,852],{},[854,965,966,968],{},[480,967,859],{"mathvariant":858},[464,969,970,972,974],{},[480,971,755],{},[850,973,866],{},[480,975,482],{},[487,977,978],{"encoding":489},"K \\in \\mathbb{R}^{m \\times d}",[47,980,982,1002],{"className":981,"ariaHidden":495},[494],[47,983,985,989,993,996,999],{"className":984},[499],[47,986],{"className":987,"style":988},[503],"height:0.7224em;vertical-align:-0.0391em;",[47,990,961],{"className":991,"style":992},[508,581],"margin-right:0.07153em;",[47,994],{"className":995,"style":889},[888],[47,997,852],{"className":998},[893],[47,1000],{"className":1001,"style":889},[888],[47,1003,1005,1008],{"className":1004},[499],[47,1006],{"className":1007,"style":903},[503],[47,1009,1011,1014],{"className":1010},[508],[47,1012,859],{"className":1013},[508,910],[47,1015,1017],{"className":1016},[585],[47,1018,1020],{"className":1019},[520],[47,1021,1023],{"className":1022},[525],[47,1024,1026],{"className":1025,"style":903},[529],[47,1027,1028,1031],{"style":925},[47,1029],{"className":1030,"style":929},[537],[47,1032,1034],{"className":1033},[542,543,544,545],[47,1035,1037,1040,1043],{"className":1036},[508,545],[47,1038,755],{"className":1039},[508,581,545],[47,1041,866],{"className":1042},[942,545],[47,1044,482],{"className":1045},[508,581,545]," 和值 ",[47,1048,1050,1079],{"className":1049},[451],[47,1051,1053],{"className":1052},[455],[457,1054,1055],{"xmlns":459},[461,1056,1057,1076],{},[464,1058,1059,1062,1064],{},[480,1060,1061],{},"V",[850,1063,852],{},[854,1065,1066,1068],{},[480,1067,859],{"mathvariant":858},[464,1069,1070,1072,1074],{},[480,1071,755],{},[850,1073,866],{},[480,1075,815],{},[487,1077,1078],{"encoding":489},"V \\in \\mathbb{R}^{m \\times v}",[47,1080,1082,1101],{"className":1081,"ariaHidden":495},[494],[47,1083,1085,1088,1092,1095,1098],{"className":1084},[499],[47,1086],{"className":1087,"style":988},[503],[47,1089,1061],{"className":1090,"style":1091},[508,581],"margin-right:0.22222em;",[47,1093],{"className":1094,"style":889},[888],[47,1096,852],{"className":1097},[893],[47,1099],{"className":1100,"style":889},[888],[47,1102,1104,1108],{"className":1103},[499],[47,1105],{"className":1106,"style":1107},[503],"height:0.7713em;",[47,1109,1111,1114],{"className":1110},[508],[47,1112,859],{"className":1113},[508,910],[47,1115,1117],{"className":1116},[585],[47,1118,1120],{"className":1119},[520],[47,1121,1123],{"className":1122},[525],[47,1124,1126],{"className":1125,"style":1107},[529],[47,1127,1128,1131],{"style":925},[47,1129],{"className":1130,"style":929},[537],[47,1132,1134],{"className":1133},[542,543,544,545],[47,1135,1137,1140,1143],{"className":1136},[508,545],[47,1138,755],{"className":1139},[508,581,545],[47,1141,866],{"className":1142},[942,545],[47,1144,815],{"className":1145,"style":830},[508,581,545]," 的缩放点积注意力是：",[14,1148,1149],{},[47,1150,1152,1255],{"className":1151},[451],[47,1153,1155],{"className":1154},[455],[457,1156,1157],{"xmlns":459},[461,1158,1159,1252],{},[464,1160,1161,1164,1167,1169,1172,1174,1176,1179,1182,1184,1188,1190,1193,1195,1197,1199,1202,1204,1208,1211,1234,1236,1238,1250],{},[480,1162,1163],{},"A",[480,1165,1166],{},"t",[480,1168,1166],{},[480,1170,1171],{},"e",[480,1173,724],{},[480,1175,1166],{},[480,1177,1178],{},"i",[480,1180,1181],{},"o",[480,1183,724],{},[850,1185,1187],{"stretchy":1186},"false","(",[480,1189,848],{},[850,1191,1192],{"separator":495},",",[480,1194,961],{},[850,1196,1192],{"separator":495},[480,1198,1061],{},[850,1200,1201],{"stretchy":1186},")",[850,1203,82],{},[480,1205,1207],{"mathvariant":1206},"normal","softmax",[850,1209,1210],{},"⁡",[464,1212,1213,1215,1232],{},[850,1214,1187],{"fence":495},[467,1216,1217,1228],{},[464,1218,1219,1221],{},[480,1220,848],{},[854,1222,1223,1225],{},[480,1224,961],{},[480,1226,1227],{"mathvariant":1206},"⊤",[474,1229,1230],{},[480,1231,482],{},[850,1233,1201],{"fence":495},[480,1235,1061],{},[850,1237,852],{},[854,1239,1240,1242],{},[480,1241,859],{"mathvariant":858},[464,1243,1244,1246,1248],{},[480,1245,724],{},[850,1247,866],{},[480,1249,815],{},[480,1251,144],{"mathvariant":1206},[487,1253,1254],{"encoding":489}," Attention(Q,K,V) = \\operatorname{softmax}\\left(\\frac{QK^\\top}{\\sqrt{d}}\\right)V \\in \\mathbb{R}^{n \\times v}.",[47,1256,1258,1328,1530],{"className":1257,"ariaHidden":495},[494],[47,1259,1261,1265,1268,1272,1275,1278,1281,1284,1287,1290,1293,1296,1300,1304,1307,1310,1313,1316,1319,1322,1325],{"className":1260},[499],[47,1262],{"className":1263,"style":1264},[503],"height:1em;vertical-align:-0.25em;",[47,1266,1163],{"className":1267},[508,581],[47,1269,1271],{"className":1270},[508,581],"tt",[47,1273,1171],{"className":1274},[508,581],[47,1276,724],{"className":1277},[508,581],[47,1279,1166],{"className":1280},[508,581],[47,1282,1178],{"className":1283},[508,581],[47,1285,1181],{"className":1286},[508,581],[47,1288,724],{"className":1289},[508,581],[47,1291,1187],{"className":1292},[512],[47,1294,848],{"className":1295},[508,581],[47,1297,1192],{"className":1298},[1299],"mpunct",[47,1301],{"className":1302,"style":1303},[888],"margin-right:0.1667em;",[47,1305,961],{"className":1306,"style":992},[508,581],[47,1308,1192],{"className":1309},[1299],[47,1311],{"className":1312,"style":1303},[888],[47,1314,1061],{"className":1315,"style":1091},[508,581],[47,1317,1201],{"className":1318},[702],[47,1320],{"className":1321,"style":889},[888],[47,1323,82],{"className":1324},[893],[47,1326],{"className":1327,"style":889},[888],[47,1329,1331,1335,1343,1346,1515,1518,1521,1524,1527],{"className":1330},[499],[47,1332],{"className":1333,"style":1334},[503],"height:1.8em;vertical-align:-0.65em;",[47,1336,1339],{"className":1337},[1338],"mop",[47,1340,1207],{"className":1341},[508,1342],"mathrm",[47,1344],{"className":1345,"style":1303},[888],[47,1347,1350,1360,1509],{"className":1348},[1349],"minner",[47,1351,1355],{"className":1352,"style":1354},[512,1353],"delimcenter","top:0em;",[47,1356,1187],{"className":1357},[1358,1359],"delimsizing","size2",[47,1361,1363,1366,1506],{"className":1362},[508],[47,1364],{"className":1365},[512,513],[47,1367,1369],{"className":1368},[467],[47,1370,1372,1498],{"className":1371},[520,521],[47,1373,1375,1495],{"className":1374},[525],[47,1376,1379,1441,1449],{"className":1377,"style":1378},[529],"height:1.095em;",[47,1380,1382,1385],{"style":1381},"top:-2.5335em;",[47,1383],{"className":1384,"style":538},[537],[47,1386,1388],{"className":1387},[542,543,544,545],[47,1389,1391],{"className":1390},[508,545],[47,1392,1394],{"className":1393},[508,552,545],[47,1395,1397,1432],{"className":1396},[520,521],[47,1398,1400,1429],{"className":1399},[525],[47,1401,1404,1416],{"className":1402,"style":1403},[529],"height:0.9378em;",[47,1405,1407,1410],{"className":1406,"style":567},[566],[47,1408],{"className":1409,"style":538},[537],[47,1411,1413],{"className":1412,"style":574},[508,545],[47,1414,482],{"className":1415},[508,581,545],[47,1417,1419,1422],{"style":1418},"top:-2.8978em;",[47,1420],{"className":1421,"style":538},[537],[47,1423,1425],{"className":1424,"style":636},[635,545],[638,1426,1427],{"xmlns":640,"width":641,"height":642,"viewBox":643,"preserveAspectRatio":644},[646,1428],{"d":648},[47,1430,616],{"className":1431},[615],[47,1433,1435],{"className":1434},[525],[47,1436,1439],{"className":1437,"style":1438},[529],"height:0.1022em;",[47,1440],{},[47,1442,1443,1446],{"style":663},[47,1444],{"className":1445,"style":538},[537],[47,1447],{"className":1448,"style":671},[670],[47,1450,1452,1455],{"style":1451},"top:-3.4461em;",[47,1453],{"className":1454,"style":538},[537],[47,1456,1458],{"className":1457},[542,543,544,545],[47,1459,1461,1464],{"className":1460},[508,545],[47,1462,848],{"className":1463},[508,581,545],[47,1465,1467,1470],{"className":1466},[508,545],[47,1468,961],{"className":1469,"style":992},[508,581,545],[47,1471,1473],{"className":1472},[585],[47,1474,1476],{"className":1475},[520],[47,1477,1479],{"className":1478},[525],[47,1480,1483],{"className":1481,"style":1482},[529],"height:0.927em;",[47,1484,1486,1489],{"style":1485},"top:-2.931em;margin-right:0.0714em;",[47,1487],{"className":1488,"style":602},[537],[47,1490,1492],{"className":1491},[542,606,607,545],[47,1493,1227],{"className":1494},[508,545],[47,1496,616],{"className":1497},[615],[47,1499,1501],{"className":1500},[525],[47,1502,1504],{"className":1503,"style":696},[529],[47,1505],{},[47,1507],{"className":1508},[702,513],[47,1510,1512],{"className":1511,"style":1354},[702,1353],[47,1513,1201],{"className":1514},[1358,1359],[47,1516],{"className":1517,"style":1303},[888],[47,1519,1061],{"className":1520,"style":1091},[508,581],[47,1522],{"className":1523,"style":889},[888],[47,1525,852],{"className":1526},[893],[47,1528],{"className":1529,"style":889},[888],[47,1531,1533,1536,1574],{"className":1532},[499],[47,1534],{"className":1535,"style":1107},[503],[47,1537,1539,1542],{"className":1538},[508],[47,1540,859],{"className":1541},[508,910],[47,1543,1545],{"className":1544},[585],[47,1546,1548],{"className":1547},[520],[47,1549,1551],{"className":1550},[525],[47,1552,1554],{"className":1553,"style":1107},[529],[47,1555,1556,1559],{"style":925},[47,1557],{"className":1558,"style":929},[537],[47,1560,1562],{"className":1561},[542,543,544,545],[47,1563,1565,1568,1571],{"className":1564},[508,545],[47,1566,724],{"className":1567},[508,581,545],[47,1569,866],{"className":1570},[942,545],[47,1572,815],{"className":1573,"style":830},[508,581,545],[47,1575,144],{"className":1576},[508],[14,1578,1579],{},[432,1580],{"alt":1581,"src":1582},"注意力","transformer/attention.png",[14,1584,1585,1586,1660,1661,1732,1733,1853],{},"缩放点积注意力的输入由维度为 ",[47,1587,1589,1607],{"className":1588},[451],[47,1590,1592],{"className":1591},[455],[457,1593,1594],{"xmlns":459},[461,1595,1596,1604],{},[464,1597,1598],{},[477,1599,1600,1602],{},[480,1601,482],{},[480,1603,485],{},[487,1605,1606],{"encoding":489},"d_k",[47,1608,1610],{"className":1609,"ariaHidden":495},[494],[47,1611,1613,1617],{"className":1612},[499],[47,1614],{"className":1615,"style":1616},[503],"height:0.8444em;vertical-align:-0.15em;",[47,1618,1620,1623],{"className":1619},[508],[47,1621,482],{"className":1622},[508,581],[47,1624,1626],{"className":1625},[585],[47,1627,1629,1651],{"className":1628},[520,521],[47,1630,1632,1648],{"className":1631},[525],[47,1633,1636],{"className":1634,"style":1635},[529],"height:0.3361em;",[47,1637,1639,1642],{"style":1638},"top:-2.55em;margin-left:0em;margin-right:0.05em;",[47,1640],{"className":1641,"style":929},[537],[47,1643,1645],{"className":1644},[542,543,544,545],[47,1646,485],{"className":1647,"style":611},[508,581,545],[47,1649,616],{"className":1650},[615],[47,1652,1654],{"className":1653},[525],[47,1655,1658],{"className":1656,"style":1657},[529],"height:0.15em;",[47,1659],{}," ​的查询和键以及维度为 ",[47,1662,1664,1682],{"className":1663},[451],[47,1665,1667],{"className":1666},[455],[457,1668,1669],{"xmlns":459},[461,1670,1671,1679],{},[464,1672,1673],{},[477,1674,1675,1677],{},[480,1676,482],{},[480,1678,815],{},[487,1680,1681],{"encoding":489},"d_v",[47,1683,1685],{"className":1684,"ariaHidden":495},[494],[47,1686,1688,1691],{"className":1687},[499],[47,1689],{"className":1690,"style":1616},[503],[47,1692,1694,1697],{"className":1693},[508],[47,1695,482],{"className":1696},[508,581],[47,1698,1700],{"className":1699},[585],[47,1701,1703,1724],{"className":1702},[520,521],[47,1704,1706,1721],{"className":1705},[525],[47,1707,1710],{"className":1708,"style":1709},[529],"height:0.1514em;",[47,1711,1712,1715],{"style":1638},[47,1713],{"className":1714,"style":929},[537],[47,1716,1718],{"className":1717},[542,543,544,545],[47,1719,815],{"className":1720,"style":830},[508,581,545],[47,1722,616],{"className":1723},[615],[47,1725,1727],{"className":1726},[525],[47,1728,1730],{"className":1729,"style":1657},[529],[47,1731],{}," ​的值组成。我们计算查询与所有键的点积，将每个点积除以 ",[47,1734,1736,1756],{"className":1735},[451],[47,1737,1739],{"className":1738},[455],[457,1740,1741],{"xmlns":459},[461,1742,1743,1753],{},[464,1744,1745],{},[474,1746,1747],{},[477,1748,1749,1751],{},[480,1750,482],{},[480,1752,485],{},[487,1754,1755],{"encoding":489},"\\sqrt{d_k}",[47,1757,1759],{"className":1758,"ariaHidden":495},[494],[47,1760,1762,1766],{"className":1761},[499],[47,1763],{"className":1764,"style":1765},[503],"height:1.04em;vertical-align:-0.1828em;",[47,1767,1769],{"className":1768},[508,552],[47,1770,1772,1844],{"className":1771},[520,521],[47,1773,1775,1841],{"className":1774},[525],[47,1776,1779,1828],{"className":1777,"style":1778},[529],"height:0.8572em;",[47,1780,1782,1785],{"className":1781,"style":567},[566],[47,1783],{"className":1784,"style":538},[537],[47,1786,1788],{"className":1787,"style":574},[508],[47,1789,1791,1794],{"className":1790},[508],[47,1792,482],{"className":1793},[508,581],[47,1795,1797],{"className":1796},[585],[47,1798,1800,1820],{"className":1799},[520,521],[47,1801,1803,1817],{"className":1802},[525],[47,1804,1806],{"className":1805,"style":1635},[529],[47,1807,1808,1811],{"style":1638},[47,1809],{"className":1810,"style":929},[537],[47,1812,1814],{"className":1813},[542,543,544,545],[47,1815,485],{"className":1816,"style":611},[508,581,545],[47,1818,616],{"className":1819},[615],[47,1821,1823],{"className":1822},[525],[47,1824,1826],{"className":1825,"style":1657},[529],[47,1827],{},[47,1829,1831,1834],{"style":1830},"top:-2.8172em;",[47,1832],{"className":1833,"style":538},[537],[47,1835,1837],{"className":1836,"style":636},[635],[638,1838,1839],{"xmlns":640,"width":641,"height":642,"viewBox":643,"preserveAspectRatio":644},[646,1840],{"d":648},[47,1842,616],{"className":1843},[615],[47,1845,1847],{"className":1846},[525],[47,1848,1851],{"className":1849,"style":1850},[529],"height:0.1828em;",[47,1852],{},"​​，并应用softmax函数以获得值上的权重。",[14,1855,1856,1857,1926,1927,1996,1997,2066,2067,2097,2098,2126,2127,2157,2158,2186,2187,2490,2491,2157,2519,2588,2589,703],{},"对于较小的 ",[47,1858,1860,1877],{"className":1859},[451],[47,1861,1863],{"className":1862},[455],[457,1864,1865],{"xmlns":459},[461,1866,1867,1875],{},[464,1868,1869],{},[477,1870,1871,1873],{},[480,1872,482],{},[480,1874,485],{},[487,1876,1606],{"encoding":489},[47,1878,1880],{"className":1879,"ariaHidden":495},[494],[47,1881,1883,1886],{"className":1882},[499],[47,1884],{"className":1885,"style":1616},[503],[47,1887,1889,1892],{"className":1888},[508],[47,1890,482],{"className":1891},[508,581],[47,1893,1895],{"className":1894},[585],[47,1896,1898,1918],{"className":1897},[520,521],[47,1899,1901,1915],{"className":1900},[525],[47,1902,1904],{"className":1903,"style":1635},[529],[47,1905,1906,1909],{"style":1638},[47,1907],{"className":1908,"style":929},[537],[47,1910,1912],{"className":1911},[542,543,544,545],[47,1913,485],{"className":1914,"style":611},[508,581,545],[47,1916,616],{"className":1917},[615],[47,1919,1921],{"className":1920},[525],[47,1922,1924],{"className":1923,"style":1657},[529],[47,1925],{}," 值来说，加性注意力和点积注意力的表现相似，但加性注意力在 ",[47,1928,1930,1947],{"className":1929},[451],[47,1931,1933],{"className":1932},[455],[457,1934,1935],{"xmlns":459},[461,1936,1937,1945],{},[464,1938,1939],{},[477,1940,1941,1943],{},[480,1942,482],{},[480,1944,485],{},[487,1946,1606],{"encoding":489},[47,1948,1950],{"className":1949,"ariaHidden":495},[494],[47,1951,1953,1956],{"className":1952},[499],[47,1954],{"className":1955,"style":1616},[503],[47,1957,1959,1962],{"className":1958},[508],[47,1960,482],{"className":1961},[508,581],[47,1963,1965],{"className":1964},[585],[47,1966,1968,1988],{"className":1967},[520,521],[47,1969,1971,1985],{"className":1970},[525],[47,1972,1974],{"className":1973,"style":1635},[529],[47,1975,1976,1979],{"style":1638},[47,1977],{"className":1978,"style":929},[537],[47,1980,1982],{"className":1981},[542,543,544,545],[47,1983,485],{"className":1984,"style":611},[508,581,545],[47,1986,616],{"className":1987},[615],[47,1989,1991],{"className":1990},[525],[47,1992,1994],{"className":1993,"style":1657},[529],[47,1995],{}," 值较大时会优于点积注意力。我们怀疑在 ",[47,1998,2000,2017],{"className":1999},[451],[47,2001,2003],{"className":2002},[455],[457,2004,2005],{"xmlns":459},[461,2006,2007,2015],{},[464,2008,2009],{},[477,2010,2011,2013],{},[480,2012,482],{},[480,2014,485],{},[487,2016,1606],{"encoding":489},[47,2018,2020],{"className":2019,"ariaHidden":495},[494],[47,2021,2023,2026],{"className":2022},[499],[47,2024],{"className":2025,"style":1616},[503],[47,2027,2029,2032],{"className":2028},[508],[47,2030,482],{"className":2031},[508,581],[47,2033,2035],{"className":2034},[585],[47,2036,2038,2058],{"className":2037},[520,521],[47,2039,2041,2055],{"className":2040},[525],[47,2042,2044],{"className":2043,"style":1635},[529],[47,2045,2046,2049],{"style":1638},[47,2047],{"className":2048,"style":929},[537],[47,2050,2052],{"className":2051},[542,543,544,545],[47,2053,485],{"className":2054,"style":611},[508,581,545],[47,2056,616],{"className":2057},[615],[47,2059,2061],{"className":2060},[525],[47,2062,2064],{"className":2063,"style":1657},[529],[47,2065],{}," 值较大时，点积的值会很大，导致 softmax 集中在很小的一个区域内。为了更好地说明这一点，我们假设查询 ",[47,2068,2070,2084],{"className":2069},[451],[47,2071,2073],{"className":2072},[455],[457,2074,2075],{"xmlns":459},[461,2076,2077,2082],{},[464,2078,2079],{},[480,2080,2081],{},"q",[487,2083,2081],{"encoding":489},[47,2085,2087],{"className":2086,"ariaHidden":495},[494],[47,2088,2090,2094],{"className":2089},[499],[47,2091],{"className":2092,"style":2093},[503],"height:0.625em;vertical-align:-0.1944em;",[47,2095,2081],{"className":2096,"style":830},[508,581]," 和键 ",[47,2099,2101,2114],{"className":2100},[451],[47,2102,2104],{"className":2103},[455],[457,2105,2106],{"xmlns":459},[461,2107,2108,2112],{},[464,2109,2110],{},[480,2111,485],{},[487,2113,485],{"encoding":489},[47,2115,2117],{"className":2116,"ariaHidden":495},[494],[47,2118,2120,2123],{"className":2119},[499],[47,2121],{"className":2122,"style":796},[503],[47,2124,485],{"className":2125,"style":611},[508,581]," 都符合一个均值为 ",[47,2128,2130,2144],{"className":2129},[451],[47,2131,2133],{"className":2132},[455],[457,2134,2135],{"xmlns":459},[461,2136,2137,2142],{},[464,2138,2139],{},[470,2140,2141],{},"0",[487,2143,2141],{"encoding":489},[47,2145,2147],{"className":2146,"ariaHidden":495},[494],[47,2148,2150,2154],{"className":2149},[499],[47,2151],{"className":2152,"style":2153},[503],"height:0.6444em;",[47,2155,2141],{"className":2156},[508],"，方差为 ",[47,2159,2161,2174],{"className":2160},[451],[47,2162,2164],{"className":2163},[455],[457,2165,2166],{"xmlns":459},[461,2167,2168,2172],{},[464,2169,2170],{},[470,2171,472],{},[487,2173,472],{"encoding":489},[47,2175,2177],{"className":2176,"ariaHidden":495},[494],[47,2178,2180,2183],{"className":2179},[499],[47,2181],{"className":2182,"style":2153},[503],[47,2184,472],{"className":2185},[508]," 的随机变量。他们点积的结果 ",[47,2188,2190,2243],{"className":2189},[451],[47,2191,2193],{"className":2192},[455],[457,2194,2195],{"xmlns":459},[461,2196,2197,2240],{},[464,2198,2199,2201,2204,2206,2208,2228,2234],{},[480,2200,2081],{},[850,2202,2203],{},"⋅",[480,2205,485],{},[850,2207,82],{},[2209,2210,2211,2214,2222],"msubsup",{},[850,2212,2213],{},"∑",[464,2215,2216,2218,2220],{},[480,2217,1178],{},[850,2219,82],{},[470,2221,472],{},[477,2223,2224,2226],{},[480,2225,482],{},[480,2227,485],{},[477,2229,2230,2232],{},[480,2231,2081],{},[480,2233,1178],{},[477,2235,2236,2238],{},[480,2237,485],{},[480,2239,1178],{},[487,2241,2242],{"encoding":489},"q \\cdot k = \\sum_{i=1}^{d_k}q_{i}k_{i}",[47,2244,2246,2266,2284],{"className":2245,"ariaHidden":495},[494],[47,2247,2249,2253,2256,2260,2263],{"className":2248},[499],[47,2250],{"className":2251,"style":2252},[503],"height:0.6389em;vertical-align:-0.1944em;",[47,2254,2081],{"className":2255,"style":830},[508,581],[47,2257],{"className":2258,"style":2259},[888],"margin-right:0.2222em;",[47,2261,2203],{"className":2262},[942],[47,2264],{"className":2265,"style":2259},[888],[47,2267,2269,2272,2275,2278,2281],{"className":2268},[499],[47,2270],{"className":2271,"style":796},[503],[47,2273,485],{"className":2274,"style":611},[508,581],[47,2276],{"className":2277,"style":889},[888],[47,2279,82],{"className":2280},[893],[47,2282],{"className":2283,"style":889},[888],[47,2285,2287,2291,2398,2401,2446],{"className":2286},[499],[47,2288],{"className":2289,"style":2290},[503],"height:1.2887em;vertical-align:-0.2997em;",[47,2292,2294,2300],{"className":2293},[1338],[47,2295,2213],{"className":2296,"style":2299},[1338,2297,2298],"op-symbol","small-op","position:relative;top:0em;",[47,2301,2303],{"className":2302},[585],[47,2304,2306,2389],{"className":2305},[520,521],[47,2307,2309,2386],{"className":2308},[525],[47,2310,2313,2334],{"className":2311,"style":2312},[529],"height:0.989em;",[47,2314,2316,2319],{"style":2315},"top:-2.4003em;margin-left:0em;margin-right:0.05em;",[47,2317],{"className":2318,"style":929},[537],[47,2320,2322],{"className":2321},[542,543,544,545],[47,2323,2325,2328,2331],{"className":2324},[508,545],[47,2326,1178],{"className":2327},[508,581,545],[47,2329,82],{"className":2330},[893,545],[47,2332,472],{"className":2333},[508,545],[47,2335,2337,2340],{"style":2336},"top:-3.2029em;margin-right:0.05em;",[47,2338],{"className":2339,"style":929},[537],[47,2341,2343],{"className":2342},[542,543,544,545],[47,2344,2346],{"className":2345},[508,545],[47,2347,2349,2352],{"className":2348},[508,545],[47,2350,482],{"className":2351},[508,581,545],[47,2353,2355],{"className":2354},[585],[47,2356,2358,2378],{"className":2357},[520,521],[47,2359,2361,2375],{"className":2360},[525],[47,2362,2364],{"className":2363,"style":595},[529],[47,2365,2366,2369],{"style":598},[47,2367],{"className":2368,"style":602},[537],[47,2370,2372],{"className":2371},[542,606,607,545],[47,2373,485],{"className":2374,"style":611},[508,581,545],[47,2376,616],{"className":2377},[615],[47,2379,2381],{"className":2380},[525],[47,2382,2384],{"className":2383,"style":623},[529],[47,2385],{},[47,2387,616],{"className":2388},[615],[47,2390,2392],{"className":2391},[525],[47,2393,2396],{"className":2394,"style":2395},[529],"height:0.2997em;",[47,2397],{},[47,2399],{"className":2400,"style":1303},[888],[47,2402,2404,2407],{"className":2403},[508],[47,2405,2081],{"className":2406,"style":830},[508,581],[47,2408,2410],{"className":2409},[585],[47,2411,2413,2438],{"className":2412},[520,521],[47,2414,2416,2435],{"className":2415},[525],[47,2417,2420],{"className":2418,"style":2419},[529],"height:0.3117em;",[47,2421,2423,2426],{"style":2422},"top:-2.55em;margin-left:-0.0359em;margin-right:0.05em;",[47,2424],{"className":2425,"style":929},[537],[47,2427,2429],{"className":2428},[542,543,544,545],[47,2430,2432],{"className":2431},[508,545],[47,2433,1178],{"className":2434},[508,581,545],[47,2436,616],{"className":2437},[615],[47,2439,2441],{"className":2440},[525],[47,2442,2444],{"className":2443,"style":1657},[529],[47,2445],{},[47,2447,2449,2452],{"className":2448},[508],[47,2450,485],{"className":2451,"style":611},[508,581],[47,2453,2455],{"className":2454},[585],[47,2456,2458,2482],{"className":2457},[520,521],[47,2459,2461,2479],{"className":2460},[525],[47,2462,2464],{"className":2463,"style":2419},[529],[47,2465,2467,2470],{"style":2466},"top:-2.55em;margin-left:-0.0315em;margin-right:0.05em;",[47,2468],{"className":2469,"style":929},[537],[47,2471,2473],{"className":2472},[542,543,544,545],[47,2474,2476],{"className":2475},[508,545],[47,2477,1178],{"className":2478},[508,581,545],[47,2480,616],{"className":2481},[615],[47,2483,2485],{"className":2484},[525],[47,2486,2488],{"className":2487,"style":1657},[529],[47,2489],{}," 的均值为 ",[47,2492,2494,2507],{"className":2493},[451],[47,2495,2497],{"className":2496},[455],[457,2498,2499],{"xmlns":459},[461,2500,2501,2505],{},[464,2502,2503],{},[470,2504,2141],{},[487,2506,2141],{"encoding":489},[47,2508,2510],{"className":2509,"ariaHidden":495},[494],[47,2511,2513,2516],{"className":2512},[499],[47,2514],{"className":2515,"style":2153},[503],[47,2517,2141],{"className":2518},[508],[47,2520,2522,2539],{"className":2521},[451],[47,2523,2525],{"className":2524},[455],[457,2526,2527],{"xmlns":459},[461,2528,2529,2537],{},[464,2530,2531],{},[477,2532,2533,2535],{},[480,2534,482],{},[480,2536,485],{},[487,2538,1606],{"encoding":489},[47,2540,2542],{"className":2541,"ariaHidden":495},[494],[47,2543,2545,2548],{"className":2544},[499],[47,2546],{"className":2547,"style":1616},[503],[47,2549,2551,2554],{"className":2550},[508],[47,2552,482],{"className":2553},[508,581],[47,2555,2557],{"className":2556},[585],[47,2558,2560,2580],{"className":2559},[520,521],[47,2561,2563,2577],{"className":2562},[525],[47,2564,2566],{"className":2565,"style":1635},[529],[47,2567,2568,2571],{"style":1638},[47,2569],{"className":2570,"style":929},[537],[47,2572,2574],{"className":2573},[542,543,544,545],[47,2575,485],{"className":2576,"style":611},[508,581,545],[47,2578,616],{"className":2579},[615],[47,2581,2583],{"className":2582},[525],[47,2584,2586],{"className":2585,"style":1657},[529],[47,2587],{},"，为了抵消这种影响，我们要乘以一个缩放因子 ",[47,2590,2592,2615],{"className":2591},[451],[47,2593,2595],{"className":2594},[455],[457,2596,2597],{"xmlns":459},[461,2598,2599,2613],{},[464,2600,2601],{},[467,2602,2603,2605],{},[470,2604,472],{},[474,2606,2607],{},[477,2608,2609,2611],{},[480,2610,482],{},[480,2612,485],{},[487,2614,490],{"encoding":489},[47,2616,2618],{"className":2617,"ariaHidden":495},[494],[47,2619,2621,2624],{"className":2620},[499],[47,2622],{"className":2623,"style":504},[503],[47,2625,2627,2630,2770],{"className":2626},[508],[47,2628],{"className":2629},[512,513],[47,2631,2633],{"className":2632},[467],[47,2634,2636,2762],{"className":2635},[520,521],[47,2637,2639,2759],{"className":2638},[525],[47,2640,2642,2737,2745],{"className":2641,"style":530},[529],[47,2643,2644,2647],{"style":533},[47,2645],{"className":2646,"style":538},[537],[47,2648,2650],{"className":2649},[542,543,544,545],[47,2651,2653],{"className":2652},[508,545],[47,2654,2656],{"className":2655},[508,552,545],[47,2657,2659,2729],{"className":2658},[520,521],[47,2660,2662,2726],{"className":2661},[525],[47,2663,2665,2714],{"className":2664,"style":562},[529],[47,2666,2668,2671],{"className":2667,"style":567},[566],[47,2669],{"className":2670,"style":538},[537],[47,2672,2674],{"className":2673,"style":574},[508,545],[47,2675,2677,2680],{"className":2676},[508,545],[47,2678,482],{"className":2679},[508,581,545],[47,2681,2683],{"className":2682},[585],[47,2684,2686,2706],{"className":2685},[520,521],[47,2687,2689,2703],{"className":2688},[525],[47,2690,2692],{"className":2691,"style":595},[529],[47,2693,2694,2697],{"style":598},[47,2695],{"className":2696,"style":602},[537],[47,2698,2700],{"className":2699},[542,606,607,545],[47,2701,485],{"className":2702,"style":611},[508,581,545],[47,2704,616],{"className":2705},[615],[47,2707,2709],{"className":2708},[525],[47,2710,2712],{"className":2711,"style":623},[529],[47,2713],{},[47,2715,2716,2719],{"style":628},[47,2717],{"className":2718,"style":538},[537],[47,2720,2722],{"className":2721,"style":636},[635,545],[638,2723,2724],{"xmlns":640,"width":641,"height":642,"viewBox":643,"preserveAspectRatio":644},[646,2725],{"d":648},[47,2727,616],{"className":2728},[615],[47,2730,2732],{"className":2731},[525],[47,2733,2735],{"className":2734,"style":658},[529],[47,2736],{},[47,2738,2739,2742],{"style":663},[47,2740],{"className":2741,"style":538},[537],[47,2743],{"className":2744,"style":671},[670],[47,2746,2747,2750],{"style":674},[47,2748],{"className":2749,"style":538},[537],[47,2751,2753],{"className":2752},[542,543,544,545],[47,2754,2756],{"className":2755},[508,545],[47,2757,472],{"className":2758},[508,545],[47,2760,616],{"className":2761},[615],[47,2763,2765],{"className":2764},[525],[47,2766,2768],{"className":2767,"style":696},[529],[47,2769],{},[47,2771],{"className":2772},[702,513],[14,2774,2775],{},"因为在训练和推理时，并非所有的值都应该加入注意力汇聚操作中，所以我们首先实现 masked_softmax 函数来保证在注意力汇聚时仅加入有意义的值。",[37,2777,2781],{"className":2778,"code":2779,"language":2780,"meta":42,"style":42},"language-python shiki shiki-themes github-light dracula","def masked_softmax(X, valid_lens):\n    \"\"\"通过在最后一个轴上掩蔽元素来执行softmax操作\"\"\"\n    # X:3D张量，valid_lens:1D或2D张量\n    if valid_lens is None:\n        return nn.functional.softmax(X, dim=-1)\n    else:\n        shape = X.shape\n        if valid_lens.dim() == 1:\n            valid_lens = torch.repeat_interleave(valid_lens, shape[1])\n        else:\n            valid_lens = valid_lens.reshape(-1)\n        # 最后一轴上被掩蔽的元素使用一个非常大的负值替换，从而其softmax输出为0\n        X = d2l.sequence_mask(X.reshape(-1, shape[-1]), valid_lens,\n                              value=-1e6)\n        return nn.functional.softmax(X.reshape(shape), dim=-1)\n","python",[44,2782,2783,2806,2812,2817,2834,2854,2862,2873,2890,2906,2914,2931,2937,2962,2975],{"__ignoreMap":42},[47,2784,2785,2788,2791,2793,2797,2800,2803],{"class":49,"line":50},[47,2786,2787],{"class":81},"def",[47,2789,2790],{"class":53}," masked_softmax",[47,2792,1187],{"class":353},[47,2794,2796],{"class":2795},"syNf4","X",[47,2798,2799],{"class":353},", ",[47,2801,2802],{"class":2795},"valid_lens",[47,2804,2805],{"class":353},"):\n",[47,2807,2808],{"class":49,"line":64},[47,2809,2811],{"class":2810},"seLWX","    \"\"\"通过在最后一个轴上掩蔽元素来执行softmax操作\"\"\"\n",[47,2813,2814],{"class":49,"line":74},[47,2815,2816],{"class":331},"    # X:3D张量，valid_lens:1D或2D张量\n",[47,2818,2819,2822,2825,2828,2831],{"class":49,"line":335},[47,2820,2821],{"class":81},"    if",[47,2823,2824],{"class":353}," valid_lens ",[47,2826,2827],{"class":81},"is",[47,2829,2830],{"class":118}," None",[47,2832,2833],{"class":353},":\n",[47,2835,2836,2839,2842,2846,2849,2851],{"class":49,"line":344},[47,2837,2838],{"class":81},"        return",[47,2840,2841],{"class":353}," nn.functional.softmax(X, ",[47,2843,2845],{"class":2844},"sQkXh","dim",[47,2847,2848],{"class":81},"=-",[47,2850,472],{"class":118},[47,2852,2853],{"class":353},")\n",[47,2855,2857,2860],{"class":49,"line":2856},6,[47,2858,2859],{"class":81},"    else",[47,2861,2833],{"class":353},[47,2863,2865,2868,2870],{"class":49,"line":2864},7,[47,2866,2867],{"class":353},"        shape ",[47,2869,82],{"class":81},[47,2871,2872],{"class":353}," X.shape\n",[47,2874,2876,2879,2882,2885,2888],{"class":49,"line":2875},8,[47,2877,2878],{"class":81},"        if",[47,2880,2881],{"class":353}," valid_lens.dim() ",[47,2883,2884],{"class":81},"==",[47,2886,2887],{"class":118}," 1",[47,2889,2833],{"class":353},[47,2891,2893,2896,2898,2901,2903],{"class":49,"line":2892},9,[47,2894,2895],{"class":353},"            valid_lens ",[47,2897,82],{"class":81},[47,2899,2900],{"class":353}," torch.repeat_interleave(valid_lens, shape[",[47,2902,472],{"class":118},[47,2904,2905],{"class":353},"])\n",[47,2907,2909,2912],{"class":49,"line":2908},10,[47,2910,2911],{"class":81},"        else",[47,2913,2833],{"class":353},[47,2915,2917,2919,2921,2924,2927,2929],{"class":49,"line":2916},11,[47,2918,2895],{"class":353},[47,2920,82],{"class":81},[47,2922,2923],{"class":353}," valid_lens.reshape(",[47,2925,2926],{"class":81},"-",[47,2928,472],{"class":118},[47,2930,2853],{"class":353},[47,2932,2934],{"class":49,"line":2933},12,[47,2935,2936],{"class":331},"        # 最后一轴上被掩蔽的元素使用一个非常大的负值替换，从而其softmax输出为0\n",[47,2938,2940,2943,2945,2948,2950,2952,2955,2957,2959],{"class":49,"line":2939},13,[47,2941,2942],{"class":353},"        X ",[47,2944,82],{"class":81},[47,2946,2947],{"class":353}," d2l.sequence_mask(X.reshape(",[47,2949,2926],{"class":81},[47,2951,472],{"class":118},[47,2953,2954],{"class":353},", shape[",[47,2956,2926],{"class":81},[47,2958,472],{"class":118},[47,2960,2961],{"class":353},"]), valid_lens,\n",[47,2963,2965,2968,2970,2973],{"class":49,"line":2964},14,[47,2966,2967],{"class":2844},"                              value",[47,2969,2848],{"class":81},[47,2971,2972],{"class":118},"1e6",[47,2974,2853],{"class":353},[47,2976,2978,2980,2983,2985,2987,2989],{"class":49,"line":2977},15,[47,2979,2838],{"class":81},[47,2981,2982],{"class":353}," nn.functional.softmax(X.reshape(shape), ",[47,2984,2845],{"class":2844},[47,2986,2848],{"class":81},[47,2988,472],{"class":118},[47,2990,2853],{"class":353},[14,2992,2993],{},"下面实现缩放点积注意力。",[37,2995,2997],{"className":2778,"code":2996,"language":2780,"meta":42,"style":42},"class DotProductAttention(nn.Module):\n    \"\"\"缩放点积注意力\"\"\"\n    def __init__(self, dropout, **kwargs):\n        super(DotProductAttention, self).__init__(**kwargs)\n        self.dropout = nn.Dropout(dropout)\n\n    # queries的形状：(batch_size，查询的个数，d)\n    # keys的形状：(batch_size，“键－值”对的个数，d)\n    # values的形状：(batch_size，“键－值”对的个数，值的维度)\n    # valid_lens的形状:(batch_size，)或者(batch_size，查询的个数)\n    def forward(self, queries, keys, values, valid_lens=None):\n        d = queries.shape[-1]\n        # 设置transpose_b=True为了交换keys的最后两个维度\n        scores = torch.bmm(queries, keys.transpose(1,2)) / math.sqrt(d)\n        self.attention_weights = masked_softmax(scores, valid_lens)\n        return torch.bmm(self.dropout(self.attention_weights), values)\n",[44,2998,2999,3021,3026,3055,3080,3093,3097,3102,3107,3112,3117,3154,3171,3176,3201,3213],{"__ignoreMap":42},[47,3000,3001,3004,3008,3010,3014,3016,3019],{"class":49,"line":50},[47,3002,3003],{"class":81},"class",[47,3005,3007],{"class":3006},"skCyd"," DotProductAttention",[47,3009,1187],{"class":353},[47,3011,3013],{"class":3012},"sDP9b","nn",[47,3015,144],{"class":353},[47,3017,3018],{"class":3012},"Module",[47,3020,2805],{"class":353},[47,3022,3023],{"class":49,"line":64},[47,3024,3025],{"class":2810},"    \"\"\"缩放点积注意力\"\"\"\n",[47,3027,3028,3031,3034,3036,3040,3042,3045,3047,3050,3053],{"class":49,"line":74},[47,3029,3030],{"class":81},"    def",[47,3032,3033],{"class":118}," __init__",[47,3035,1187],{"class":353},[47,3037,3039],{"class":3038},"sD3jg","self",[47,3041,2799],{"class":353},[47,3043,3044],{"class":2795},"dropout",[47,3046,2799],{"class":353},[47,3048,3049],{"class":81},"**",[47,3051,3052],{"class":2795},"kwargs",[47,3054,2805],{"class":353},[47,3056,3057,3061,3064,3067,3070,3073,3075,3077],{"class":49,"line":335},[47,3058,3060],{"class":3059},"sPGBF","        super",[47,3062,3063],{"class":353},"(DotProductAttention, ",[47,3065,3039],{"class":3066},"sJti5",[47,3068,3069],{"class":353},").",[47,3071,3072],{"class":118},"__init__",[47,3074,1187],{"class":353},[47,3076,3049],{"class":81},[47,3078,3079],{"class":353},"kwargs)\n",[47,3081,3082,3085,3088,3090],{"class":49,"line":344},[47,3083,3084],{"class":3066},"        self",[47,3086,3087],{"class":353},".dropout ",[47,3089,82],{"class":81},[47,3091,3092],{"class":353}," nn.Dropout(dropout)\n",[47,3094,3095],{"class":49,"line":2856},[47,3096,326],{"emptyLinePlaceholder":325},[47,3098,3099],{"class":49,"line":2864},[47,3100,3101],{"class":331},"    # queries的形状：(batch_size，查询的个数，d)\n",[47,3103,3104],{"class":49,"line":2875},[47,3105,3106],{"class":331},"    # keys的形状：(batch_size，“键－值”对的个数，d)\n",[47,3108,3109],{"class":49,"line":2892},[47,3110,3111],{"class":331},"    # values的形状：(batch_size，“键－值”对的个数，值的维度)\n",[47,3113,3114],{"class":49,"line":2908},[47,3115,3116],{"class":331},"    # valid_lens的形状:(batch_size，)或者(batch_size，查询的个数)\n",[47,3118,3119,3121,3124,3126,3128,3130,3133,3135,3138,3140,3143,3145,3147,3149,3152],{"class":49,"line":2916},[47,3120,3030],{"class":81},[47,3122,3123],{"class":53}," forward",[47,3125,1187],{"class":353},[47,3127,3039],{"class":3038},[47,3129,2799],{"class":353},[47,3131,3132],{"class":2795},"queries",[47,3134,2799],{"class":353},[47,3136,3137],{"class":2795},"keys",[47,3139,2799],{"class":353},[47,3141,3142],{"class":2795},"values",[47,3144,2799],{"class":353},[47,3146,2802],{"class":2795},[47,3148,82],{"class":81},[47,3150,3151],{"class":118},"None",[47,3153,2805],{"class":353},[47,3155,3156,3159,3161,3164,3166,3168],{"class":49,"line":2933},[47,3157,3158],{"class":353},"        d ",[47,3160,82],{"class":81},[47,3162,3163],{"class":353}," queries.shape[",[47,3165,2926],{"class":81},[47,3167,472],{"class":118},[47,3169,3170],{"class":353},"]\n",[47,3172,3173],{"class":49,"line":2939},[47,3174,3175],{"class":331},"        # 设置transpose_b=True为了交换keys的最后两个维度\n",[47,3177,3178,3181,3183,3186,3188,3190,3193,3196,3198],{"class":49,"line":2964},[47,3179,3180],{"class":353},"        scores ",[47,3182,82],{"class":81},[47,3184,3185],{"class":353}," torch.bmm(queries, keys.transpose(",[47,3187,472],{"class":118},[47,3189,1192],{"class":353},[47,3191,3192],{"class":118},"2",[47,3194,3195],{"class":353},")) ",[47,3197,141],{"class":81},[47,3199,3200],{"class":353}," math.sqrt(d)\n",[47,3202,3203,3205,3208,3210],{"class":49,"line":2977},[47,3204,3084],{"class":3066},[47,3206,3207],{"class":353},".attention_weights ",[47,3209,82],{"class":81},[47,3211,3212],{"class":353}," masked_softmax(scores, valid_lens)\n",[47,3214,3216,3218,3221,3223,3226,3228],{"class":49,"line":3215},16,[47,3217,2838],{"class":81},[47,3219,3220],{"class":353}," torch.bmm(",[47,3222,3039],{"class":3066},[47,3224,3225],{"class":353},".dropout(",[47,3227,3039],{"class":3066},[47,3229,3230],{"class":353},".attention_weights), values)\n",[14,3232,3233,3234,3237],{},"代码中函数的 ",[44,3235,3236],{},"**kwargs","表示任意关键字参数，为了符合 Pytorch 的继承约定。",[207,3239,3240],{"id":3240},"多头注意力",[14,3242,3243],{},"多头注意力机制允许模型在不同位置上同时关注来自不同表示子空间的信息。使用单一注意力头时，平均操作会抑制这种能力。",[14,3245,3246],{},[432,3247],{"alt":42,"src":3248},"transformer/MHA.png",[14,3250,3251],{},[47,3252,3254,3370],{"className":3253},[451],[47,3255,3257],{"className":3256},[455],[457,3258,3259],{"xmlns":459},[461,3260,3261,3367],{},[464,3262,3263,3287,3289,3291,3293,3295,3297,3299,3301,3303,3319,3321,3336,3338,3341,3343,3357,3359],{},[464,3264,3265,3268,3271,3274,3276,3278,3281,3283,3285],{},[480,3266,3267],{"mathvariant":1206},"M",[480,3269,3270],{"mathvariant":1206},"u",[480,3272,3273],{"mathvariant":1206},"l",[480,3275,1166],{"mathvariant":1206},[480,3277,1178],{"mathvariant":1206},[480,3279,3280],{"mathvariant":1206},"H",[480,3282,1171],{"mathvariant":1206},[480,3284,285],{"mathvariant":1206},[480,3286,482],{"mathvariant":1206},[850,3288,1187],{"stretchy":1186},[480,3290,848],{},[850,3292,1192],{"separator":495},[480,3294,961],{},[850,3296,1192],{"separator":495},[480,3298,1061],{},[850,3300,1201],{"stretchy":1186},[850,3302,82],{},[464,3304,3305,3308,3310,3312,3315,3317],{},[480,3306,3307],{"mathvariant":1206},"C",[480,3309,1181],{"mathvariant":1206},[480,3311,724],{"mathvariant":1206},[480,3313,3314],{"mathvariant":1206},"c",[480,3316,285],{"mathvariant":1206},[480,3318,1166],{"mathvariant":1206},[850,3320,1187],{"stretchy":1186},[477,3322,3323,3334],{},[464,3324,3325,3328,3330,3332],{},[480,3326,3327],{"mathvariant":1206},"h",[480,3329,1171],{"mathvariant":1206},[480,3331,285],{"mathvariant":1206},[480,3333,482],{"mathvariant":1206},[470,3335,472],{},[850,3337,1192],{"separator":495},[850,3339,3340],{},"…",[850,3342,1192],{"separator":495},[477,3344,3345,3355],{},[464,3346,3347,3349,3351,3353],{},[480,3348,3327],{"mathvariant":1206},[480,3350,1171],{"mathvariant":1206},[480,3352,285],{"mathvariant":1206},[480,3354,482],{"mathvariant":1206},[480,3356,3327],{},[850,3358,1201],{"stretchy":1186},[854,3360,3361,3364],{},[480,3362,3363],{},"W",[480,3365,3366],{},"O",[487,3368,3369],{"encoding":489},"\\mathrm{MultiHead}(Q, K, V) = \\mathrm{Concat}(\\mathrm{head}_1, \\ldots, \\mathrm{head}_h) W^O",[47,3371,3373,3422],{"className":3372,"ariaHidden":495},[494],[47,3374,3376,3379,3386,3389,3392,3395,3398,3401,3404,3407,3410,3413,3416,3419],{"className":3375},[499],[47,3377],{"className":3378,"style":1264},[503],[47,3380,3382],{"className":3381},[508],[47,3383,3385],{"className":3384},[508,1342],"MultiHead",[47,3387,1187],{"className":3388},[512],[47,3390,848],{"className":3391},[508,581],[47,3393,1192],{"className":3394},[1299],[47,3396],{"className":3397,"style":1303},[888],[47,3399,961],{"className":3400,"style":992},[508,581],[47,3402,1192],{"className":3403},[1299],[47,3405],{"className":3406,"style":1303},[888],[47,3408,1061],{"className":3409,"style":1091},[508,581],[47,3411,1201],{"className":3412},[702],[47,3414],{"className":3415,"style":889},[888],[47,3417,82],{"className":3418},[893],[47,3420],{"className":3421,"style":889},[888],[47,3423,3425,3429,3436,3439,3485,3488,3491,3494,3497,3500,3503,3546,3549],{"className":3424},[499],[47,3426],{"className":3427,"style":3428},[503],"height:1.0913em;vertical-align:-0.25em;",[47,3430,3432],{"className":3431},[508],[47,3433,3435],{"className":3434},[508,1342],"Concat",[47,3437,1187],{"className":3438},[512],[47,3440,3442,3449],{"className":3441},[508],[47,3443,3445],{"className":3444},[508],[47,3446,3448],{"className":3447},[508,1342],"head",[47,3450,3452],{"className":3451},[585],[47,3453,3455,3477],{"className":3454},[520,521],[47,3456,3458,3474],{"className":3457},[525],[47,3459,3462],{"className":3460,"style":3461},[529],"height:0.3011em;",[47,3463,3465,3468],{"style":3464},"top:-2.55em;margin-right:0.05em;",[47,3466],{"className":3467,"style":929},[537],[47,3469,3471],{"className":3470},[542,543,544,545],[47,3472,472],{"className":3473},[508,545],[47,3475,616],{"className":3476},[615],[47,3478,3480],{"className":3479},[525],[47,3481,3483],{"className":3482,"style":1657},[529],[47,3484],{},[47,3486,1192],{"className":3487},[1299],[47,3489],{"className":3490,"style":1303},[888],[47,3492,3340],{"className":3493},[1349],[47,3495],{"className":3496,"style":1303},[888],[47,3498,1192],{"className":3499},[1299],[47,3501],{"className":3502,"style":1303},[888],[47,3504,3506,3512],{"className":3505},[508],[47,3507,3509],{"className":3508},[508],[47,3510,3448],{"className":3511},[508,1342],[47,3513,3515],{"className":3514},[585],[47,3516,3518,3538],{"className":3517},[520,521],[47,3519,3521,3535],{"className":3520},[525],[47,3522,3524],{"className":3523,"style":1635},[529],[47,3525,3526,3529],{"style":3464},[47,3527],{"className":3528,"style":929},[537],[47,3530,3532],{"className":3531},[542,543,544,545],[47,3533,3327],{"className":3534},[508,581,545],[47,3536,616],{"className":3537},[615],[47,3539,3541],{"className":3540},[525],[47,3542,3544],{"className":3543,"style":1657},[529],[47,3545],{},[47,3547,1201],{"className":3548},[702],[47,3550,3552,3556],{"className":3551},[508],[47,3553,3363],{"className":3554,"style":3555},[508,581],"margin-right:0.13889em;",[47,3557,3559],{"className":3558},[585],[47,3560,3562],{"className":3561},[520],[47,3563,3565],{"className":3564},[525],[47,3566,3569],{"className":3567,"style":3568},[529],"height:0.8413em;",[47,3570,3571,3574],{"style":925},[47,3572],{"className":3573,"style":929},[537],[47,3575,3577],{"className":3576},[542,543,544,545],[47,3578,3366],{"className":3579,"style":3580},[508,581,545],"margin-right:0.02778em;",[14,3582,3583,3584,3946,3947,4196,4197,4196,4446,4697,4698,144],{},"其中 ",[47,3585,3587,3679],{"className":3586},[451],[47,3588,3590],{"className":3589},[455],[457,3591,3592],{"xmlns":459},[461,3593,3594,3676],{},[464,3595,3596,3610,3612,3632,3634,3636,3644,3646,3650,3652,3660,3662,3664,3666,3674],{},[477,3597,3598,3608],{},[464,3599,3600,3602,3604,3606],{},[480,3601,3327],{"mathvariant":1206},[480,3603,1171],{"mathvariant":1206},[480,3605,285],{"mathvariant":1206},[480,3607,482],{"mathvariant":1206},[480,3609,1178],{},[850,3611,82],{},[464,3613,3614,3616,3618,3620,3622,3624,3626,3628,3630],{},[480,3615,1163],{"mathvariant":1206},[480,3617,1166],{"mathvariant":1206},[480,3619,1166],{"mathvariant":1206},[480,3621,1171],{"mathvariant":1206},[480,3623,724],{"mathvariant":1206},[480,3625,1166],{"mathvariant":1206},[480,3627,1178],{"mathvariant":1206},[480,3629,1181],{"mathvariant":1206},[480,3631,724],{"mathvariant":1206},[850,3633,1187],{"stretchy":1186},[480,3635,848],{},[2209,3637,3638,3640,3642],{},[480,3639,3363],{},[480,3641,1178],{},[480,3643,848],{},[850,3645,1192],{"separator":495},[3647,3648,3649],"mtext",{},"  ",[480,3651,961],{},[2209,3653,3654,3656,3658],{},[480,3655,3363],{},[480,3657,1178],{},[480,3659,961],{},[850,3661,1192],{"separator":495},[3647,3663,3649],{},[480,3665,1061],{},[2209,3667,3668,3670,3672],{},[480,3669,3363],{},[480,3671,1178],{},[480,3673,1061],{},[850,3675,1201],{"stretchy":1186},[487,3677,3678],{"encoding":489},"\\mathrm{head}_i=\\mathrm{Attention}(QW_i^Q,\\; KW_i^K,\\; VW_i^V)",[47,3680,3682,3740],{"className":3681,"ariaHidden":495},[494],[47,3683,3685,3688,3731,3734,3737],{"className":3684},[499],[47,3686],{"className":3687,"style":1616},[503],[47,3689,3691,3697],{"className":3690},[508],[47,3692,3694],{"className":3693},[508],[47,3695,3448],{"className":3696},[508,1342],[47,3698,3700],{"className":3699},[585],[47,3701,3703,3723],{"className":3702},[520,521],[47,3704,3706,3720],{"className":3705},[525],[47,3707,3709],{"className":3708,"style":2419},[529],[47,3710,3711,3714],{"style":3464},[47,3712],{"className":3713,"style":929},[537],[47,3715,3717],{"className":3716},[542,543,544,545],[47,3718,1178],{"className":3719},[508,581,545],[47,3721,616],{"className":3722},[615],[47,3724,3726],{"className":3725},[525],[47,3727,3729],{"className":3728,"style":1657},[529],[47,3730],{},[47,3732],{"className":3733,"style":889},[888],[47,3735,82],{"className":3736},[893],[47,3738],{"className":3739,"style":889},[888],[47,3741,3743,3747,3754,3757,3760,3815,3818,3821,3824,3827,3880,3883,3886,3889,3892,3943],{"className":3742},[499],[47,3744],{"className":3745,"style":3746},[503],"height:1.2361em;vertical-align:-0.2769em;",[47,3748,3750],{"className":3749},[508],[47,3751,3753],{"className":3752},[508,1342],"Attention",[47,3755,1187],{"className":3756},[512],[47,3758,848],{"className":3759},[508,581],[47,3761,3763,3766],{"className":3762},[508],[47,3764,3363],{"className":3765,"style":3555},[508,581],[47,3767,3769],{"className":3768},[585],[47,3770,3772,3806],{"className":3771},[520,521],[47,3773,3775,3803],{"className":3774},[525],[47,3776,3779,3791],{"className":3777,"style":3778},[529],"height:0.9592em;",[47,3780,3782,3785],{"style":3781},"top:-2.4231em;margin-left:-0.1389em;margin-right:0.05em;",[47,3783],{"className":3784,"style":929},[537],[47,3786,3788],{"className":3787},[542,543,544,545],[47,3789,1178],{"className":3790},[508,581,545],[47,3792,3794,3797],{"style":3793},"top:-3.1809em;margin-right:0.05em;",[47,3795],{"className":3796,"style":929},[537],[47,3798,3800],{"className":3799},[542,543,544,545],[47,3801,848],{"className":3802},[508,581,545],[47,3804,616],{"className":3805},[615],[47,3807,3809],{"className":3808},[525],[47,3810,3813],{"className":3811,"style":3812},[529],"height:0.2769em;",[47,3814],{},[47,3816,1192],{"className":3817},[1299],[47,3819],{"className":3820,"style":889},[888],[47,3822],{"className":3823,"style":1303},[888],[47,3825,961],{"className":3826,"style":992},[508,581],[47,3828,3830,3833],{"className":3829},[508],[47,3831,3363],{"className":3832,"style":3555},[508,581],[47,3834,3836],{"className":3835},[585],[47,3837,3839,3871],{"className":3838},[520,521],[47,3840,3842,3868],{"className":3841},[525],[47,3843,3845,3857],{"className":3844,"style":3568},[529],[47,3846,3848,3851],{"style":3847},"top:-2.4413em;margin-left:-0.1389em;margin-right:0.05em;",[47,3849],{"className":3850,"style":929},[537],[47,3852,3854],{"className":3853},[542,543,544,545],[47,3855,1178],{"className":3856},[508,581,545],[47,3858,3859,3862],{"style":925},[47,3860],{"className":3861,"style":929},[537],[47,3863,3865],{"className":3864},[542,543,544,545],[47,3866,961],{"className":3867,"style":992},[508,581,545],[47,3869,616],{"className":3870},[615],[47,3872,3874],{"className":3873},[525],[47,3875,3878],{"className":3876,"style":3877},[529],"height:0.2587em;",[47,3879],{},[47,3881,1192],{"className":3882},[1299],[47,3884],{"className":3885,"style":889},[888],[47,3887],{"className":3888,"style":1303},[888],[47,3890,1061],{"className":3891,"style":1091},[508,581],[47,3893,3895,3898],{"className":3894},[508],[47,3896,3363],{"className":3897,"style":3555},[508,581],[47,3899,3901],{"className":3900},[585],[47,3902,3904,3935],{"className":3903},[520,521],[47,3905,3907,3932],{"className":3906},[525],[47,3908,3910,3921],{"className":3909,"style":3568},[529],[47,3911,3912,3915],{"style":3847},[47,3913],{"className":3914,"style":929},[537],[47,3916,3918],{"className":3917},[542,543,544,545],[47,3919,1178],{"className":3920},[508,581,545],[47,3922,3923,3926],{"style":925},[47,3924],{"className":3925,"style":929},[537],[47,3927,3929],{"className":3928},[542,543,544,545],[47,3930,1061],{"className":3931,"style":1091},[508,581,545],[47,3933,616],{"className":3934},[615],[47,3936,3938],{"className":3937},[525],[47,3939,3941],{"className":3940,"style":3877},[529],[47,3942],{},[47,3944,1201],{"className":3945},[702],"，投影参数是矩阵\n",[47,3948,3950,4002],{"className":3949},[451],[47,3951,3953],{"className":3952},[455],[457,3954,3955],{"xmlns":459},[461,3956,3957,3999],{},[464,3958,3959,3967,3969],{},[2209,3960,3961,3963,3965],{},[480,3962,3363],{},[480,3964,1178],{},[480,3966,848],{},[850,3968,852],{},[854,3970,3971,3973],{},[480,3972,859],{"mathvariant":858},[464,3974,3975,3991,3993],{},[477,3976,3977,3979],{},[480,3978,482],{},[464,3980,3981,3983,3985,3987,3989],{},[480,3982,755],{"mathvariant":1206},[480,3984,1181],{"mathvariant":1206},[480,3986,482],{"mathvariant":1206},[480,3988,1171],{"mathvariant":1206},[480,3990,3273],{"mathvariant":1206},[850,3992,866],{},[477,3994,3995,3997],{},[480,3996,482],{},[480,3998,485],{},[487,4000,4001],{"encoding":489},"W_i^Q \\in \\mathbb{R}^{d_{\\mathrm{model}} \\times d_k}",[47,4003,4005,4071],{"className":4004,"ariaHidden":495},[494],[47,4006,4008,4011,4062,4065,4068],{"className":4007},[499],[47,4009],{"className":4010,"style":3746},[503],[47,4012,4014,4017],{"className":4013},[508],[47,4015,3363],{"className":4016,"style":3555},[508,581],[47,4018,4020],{"className":4019},[585],[47,4021,4023,4054],{"className":4022},[520,521],[47,4024,4026,4051],{"className":4025},[525],[47,4027,4029,4040],{"className":4028,"style":3778},[529],[47,4030,4031,4034],{"style":3781},[47,4032],{"className":4033,"style":929},[537],[47,4035,4037],{"className":4036},[542,543,544,545],[47,4038,1178],{"className":4039},[508,581,545],[47,4041,4042,4045],{"style":3793},[47,4043],{"className":4044,"style":929},[537],[47,4046,4048],{"className":4047},[542,543,544,545],[47,4049,848],{"className":4050},[508,581,545],[47,4052,616],{"className":4053},[615],[47,4055,4057],{"className":4056},[525],[47,4058,4060],{"className":4059,"style":3812},[529],[47,4061],{},[47,4063],{"className":4064,"style":889},[888],[47,4066,852],{"className":4067},[893],[47,4069],{"className":4070,"style":889},[888],[47,4072,4074,4077],{"className":4073},[499],[47,4075],{"className":4076,"style":903},[503],[47,4078,4080,4083],{"className":4079},[508],[47,4081,859],{"className":4082},[508,910],[47,4084,4086],{"className":4085},[585],[47,4087,4089],{"className":4088},[520],[47,4090,4092],{"className":4091},[525],[47,4093,4095],{"className":4094,"style":903},[529],[47,4096,4097,4100],{"style":925},[47,4098],{"className":4099,"style":929},[537],[47,4101,4103],{"className":4102},[542,543,544,545],[47,4104,4106,4153,4156],{"className":4105},[508,545],[47,4107,4109,4112],{"className":4108},[508,545],[47,4110,482],{"className":4111},[508,581,545],[47,4113,4115],{"className":4114},[585],[47,4116,4118,4145],{"className":4117},[520,521],[47,4119,4121,4142],{"className":4120},[525],[47,4122,4124],{"className":4123,"style":595},[529],[47,4125,4126,4129],{"style":598},[47,4127],{"className":4128,"style":602},[537],[47,4130,4132],{"className":4131},[542,606,607,545],[47,4133,4135],{"className":4134},[508,545],[47,4136,4138],{"className":4137},[508,545],[47,4139,4141],{"className":4140},[508,1342,545],"model",[47,4143,616],{"className":4144},[615],[47,4146,4148],{"className":4147},[525],[47,4149,4151],{"className":4150,"style":623},[529],[47,4152],{},[47,4154,866],{"className":4155},[942,545],[47,4157,4159,4162],{"className":4158},[508,545],[47,4160,482],{"className":4161},[508,581,545],[47,4163,4165],{"className":4164},[585],[47,4166,4168,4188],{"className":4167},[520,521],[47,4169,4171,4185],{"className":4170},[525],[47,4172,4174],{"className":4173,"style":595},[529],[47,4175,4176,4179],{"style":598},[47,4177],{"className":4178,"style":602},[537],[47,4180,4182],{"className":4181},[542,606,607,545],[47,4183,485],{"className":4184,"style":611},[508,581,545],[47,4186,616],{"className":4187},[615],[47,4189,4191],{"className":4190},[525],[47,4192,4194],{"className":4193,"style":623},[529],[47,4195],{},",\n",[47,4198,4200,4252],{"className":4199},[451],[47,4201,4203],{"className":4202},[455],[457,4204,4205],{"xmlns":459},[461,4206,4207,4249],{},[464,4208,4209,4217,4219],{},[2209,4210,4211,4213,4215],{},[480,4212,3363],{},[480,4214,1178],{},[480,4216,961],{},[850,4218,852],{},[854,4220,4221,4223],{},[480,4222,859],{"mathvariant":858},[464,4224,4225,4241,4243],{},[477,4226,4227,4229],{},[480,4228,482],{},[464,4230,4231,4233,4235,4237,4239],{},[480,4232,755],{"mathvariant":1206},[480,4234,1181],{"mathvariant":1206},[480,4236,482],{"mathvariant":1206},[480,4238,1171],{"mathvariant":1206},[480,4240,3273],{"mathvariant":1206},[850,4242,866],{},[477,4244,4245,4247],{},[480,4246,482],{},[480,4248,485],{},[487,4250,4251],{"encoding":489},"W_i^K \\in \\mathbb{R}^{d_{\\mathrm{model}} \\times d_k}",[47,4253,4255,4322],{"className":4254,"ariaHidden":495},[494],[47,4256,4258,4262,4313,4316,4319],{"className":4257},[499],[47,4259],{"className":4260,"style":4261},[503],"height:1.1em;vertical-align:-0.2587em;",[47,4263,4265,4268],{"className":4264},[508],[47,4266,3363],{"className":4267,"style":3555},[508,581],[47,4269,4271],{"className":4270},[585],[47,4272,4274,4305],{"className":4273},[520,521],[47,4275,4277,4302],{"className":4276},[525],[47,4278,4280,4291],{"className":4279,"style":3568},[529],[47,4281,4282,4285],{"style":3847},[47,4283],{"className":4284,"style":929},[537],[47,4286,4288],{"className":4287},[542,543,544,545],[47,4289,1178],{"className":4290},[508,581,545],[47,4292,4293,4296],{"style":925},[47,4294],{"className":4295,"style":929},[537],[47,4297,4299],{"className":4298},[542,543,544,545],[47,4300,961],{"className":4301,"style":992},[508,581,545],[47,4303,616],{"className":4304},[615],[47,4306,4308],{"className":4307},[525],[47,4309,4311],{"className":4310,"style":3877},[529],[47,4312],{},[47,4314],{"className":4315,"style":889},[888],[47,4317,852],{"className":4318},[893],[47,4320],{"className":4321,"style":889},[888],[47,4323,4325,4328],{"className":4324},[499],[47,4326],{"className":4327,"style":903},[503],[47,4329,4331,4334],{"className":4330},[508],[47,4332,859],{"className":4333},[508,910],[47,4335,4337],{"className":4336},[585],[47,4338,4340],{"className":4339},[520],[47,4341,4343],{"className":4342},[525],[47,4344,4346],{"className":4345,"style":903},[529],[47,4347,4348,4351],{"style":925},[47,4349],{"className":4350,"style":929},[537],[47,4352,4354],{"className":4353},[542,543,544,545],[47,4355,4357,4403,4406],{"className":4356},[508,545],[47,4358,4360,4363],{"className":4359},[508,545],[47,4361,482],{"className":4362},[508,581,545],[47,4364,4366],{"className":4365},[585],[47,4367,4369,4395],{"className":4368},[520,521],[47,4370,4372,4392],{"className":4371},[525],[47,4373,4375],{"className":4374,"style":595},[529],[47,4376,4377,4380],{"style":598},[47,4378],{"className":4379,"style":602},[537],[47,4381,4383],{"className":4382},[542,606,607,545],[47,4384,4386],{"className":4385},[508,545],[47,4387,4389],{"className":4388},[508,545],[47,4390,4141],{"className":4391},[508,1342,545],[47,4393,616],{"className":4394},[615],[47,4396,4398],{"className":4397},[525],[47,4399,4401],{"className":4400,"style":623},[529],[47,4402],{},[47,4404,866],{"className":4405},[942,545],[47,4407,4409,4412],{"className":4408},[508,545],[47,4410,482],{"className":4411},[508,581,545],[47,4413,4415],{"className":4414},[585],[47,4416,4418,4438],{"className":4417},[520,521],[47,4419,4421,4435],{"className":4420},[525],[47,4422,4424],{"className":4423,"style":595},[529],[47,4425,4426,4429],{"style":598},[47,4427],{"className":4428,"style":602},[537],[47,4430,4432],{"className":4431},[542,606,607,545],[47,4433,485],{"className":4434,"style":611},[508,581,545],[47,4436,616],{"className":4437},[615],[47,4439,4441],{"className":4440},[525],[47,4442,4444],{"className":4443,"style":623},[529],[47,4445],{},[47,4447,4449,4501],{"className":4448},[451],[47,4450,4452],{"className":4451},[455],[457,4453,4454],{"xmlns":459},[461,4455,4456,4498],{},[464,4457,4458,4466,4468],{},[2209,4459,4460,4462,4464],{},[480,4461,3363],{},[480,4463,1178],{},[480,4465,1061],{},[850,4467,852],{},[854,4469,4470,4472],{},[480,4471,859],{"mathvariant":858},[464,4473,4474,4490,4492],{},[477,4475,4476,4478],{},[480,4477,482],{},[464,4479,4480,4482,4484,4486,4488],{},[480,4481,755],{"mathvariant":1206},[480,4483,1181],{"mathvariant":1206},[480,4485,482],{"mathvariant":1206},[480,4487,1171],{"mathvariant":1206},[480,4489,3273],{"mathvariant":1206},[850,4491,866],{},[477,4493,4494,4496],{},[480,4495,482],{},[480,4497,815],{},[487,4499,4500],{"encoding":489},"W_i^V \\in \\mathbb{R}^{d_{\\mathrm{model}} \\times d_v}",[47,4502,4504,4570],{"className":4503,"ariaHidden":495},[494],[47,4505,4507,4510,4561,4564,4567],{"className":4506},[499],[47,4508],{"className":4509,"style":4261},[503],[47,4511,4513,4516],{"className":4512},[508],[47,4514,3363],{"className":4515,"style":3555},[508,581],[47,4517,4519],{"className":4518},[585],[47,4520,4522,4553],{"className":4521},[520,521],[47,4523,4525,4550],{"className":4524},[525],[47,4526,4528,4539],{"className":4527,"style":3568},[529],[47,4529,4530,4533],{"style":3847},[47,4531],{"className":4532,"style":929},[537],[47,4534,4536],{"className":4535},[542,543,544,545],[47,4537,1178],{"className":4538},[508,581,545],[47,4540,4541,4544],{"style":925},[47,4542],{"className":4543,"style":929},[537],[47,4545,4547],{"className":4546},[542,543,544,545],[47,4548,1061],{"className":4549,"style":1091},[508,581,545],[47,4551,616],{"className":4552},[615],[47,4554,4556],{"className":4555},[525],[47,4557,4559],{"className":4558,"style":3877},[529],[47,4560],{},[47,4562],{"className":4563,"style":889},[888],[47,4565,852],{"className":4566},[893],[47,4568],{"className":4569,"style":889},[888],[47,4571,4573,4576],{"className":4572},[499],[47,4574],{"className":4575,"style":903},[503],[47,4577,4579,4582],{"className":4578},[508],[47,4580,859],{"className":4581},[508,910],[47,4583,4585],{"className":4584},[585],[47,4586,4588],{"className":4587},[520],[47,4589,4591],{"className":4590},[525],[47,4592,4594],{"className":4593,"style":903},[529],[47,4595,4596,4599],{"style":925},[47,4597],{"className":4598,"style":929},[537],[47,4600,4602],{"className":4601},[542,543,544,545],[47,4603,4605,4651,4654],{"className":4604},[508,545],[47,4606,4608,4611],{"className":4607},[508,545],[47,4609,482],{"className":4610},[508,581,545],[47,4612,4614],{"className":4613},[585],[47,4615,4617,4643],{"className":4616},[520,521],[47,4618,4620,4640],{"className":4619},[525],[47,4621,4623],{"className":4622,"style":595},[529],[47,4624,4625,4628],{"style":598},[47,4626],{"className":4627,"style":602},[537],[47,4629,4631],{"className":4630},[542,606,607,545],[47,4632,4634],{"className":4633},[508,545],[47,4635,4637],{"className":4636},[508,545],[47,4638,4141],{"className":4639},[508,1342,545],[47,4641,616],{"className":4642},[615],[47,4644,4646],{"className":4645},[525],[47,4647,4649],{"className":4648,"style":623},[529],[47,4650],{},[47,4652,866],{"className":4653},[942,545],[47,4655,4657,4660],{"className":4656},[508,545],[47,4658,482],{"className":4659},[508,581,545],[47,4661,4663],{"className":4662},[585],[47,4664,4666,4688],{"className":4665},[520,521],[47,4667,4669,4685],{"className":4668},[525],[47,4670,4673],{"className":4671,"style":4672},[529],"height:0.1645em;",[47,4674,4676,4679],{"style":4675},"top:-2.357em;margin-left:0em;margin-right:0.0714em;",[47,4677],{"className":4678,"style":602},[537],[47,4680,4682],{"className":4681},[542,606,607,545],[47,4683,815],{"className":4684,"style":830},[508,581,545],[47,4686,616],{"className":4687},[615],[47,4689,4691],{"className":4690},[525],[47,4692,4695],{"className":4693,"style":4694},[529],"height:0.143em;",[47,4696],{},"\n和\n",[47,4699,4701,4753],{"className":4700},[451],[47,4702,4704],{"className":4703},[455],[457,4705,4706],{"xmlns":459},[461,4707,4708,4750],{},[464,4709,4710,4716,4718],{},[854,4711,4712,4714],{},[480,4713,3363],{},[480,4715,3366],{},[850,4717,852],{},[854,4719,4720,4722],{},[480,4721,859],{"mathvariant":858},[464,4723,4724,4726,4732,4734],{},[480,4725,3327],{},[477,4727,4728,4730],{},[480,4729,482],{},[480,4731,815],{},[850,4733,866],{},[477,4735,4736,4738],{},[480,4737,482],{},[464,4739,4740,4742,4744,4746,4748],{},[480,4741,755],{"mathvariant":1206},[480,4743,1181],{"mathvariant":1206},[480,4745,482],{"mathvariant":1206},[480,4747,1171],{"mathvariant":1206},[480,4749,3273],{"mathvariant":1206},[487,4751,4752],{"encoding":489},"W^O \\in \\mathbb{R}^{h d_v \\times d_{\\mathrm{model}}}",[47,4754,4756,4801],{"className":4755,"ariaHidden":495},[494],[47,4757,4759,4763,4792,4795,4798],{"className":4758},[499],[47,4760],{"className":4761,"style":4762},[503],"height:0.8804em;vertical-align:-0.0391em;",[47,4764,4766,4769],{"className":4765},[508],[47,4767,3363],{"className":4768,"style":3555},[508,581],[47,4770,4772],{"className":4771},[585],[47,4773,4775],{"className":4774},[520],[47,4776,4778],{"className":4777},[525],[47,4779,4781],{"className":4780,"style":3568},[529],[47,4782,4783,4786],{"style":925},[47,4784],{"className":4785,"style":929},[537],[47,4787,4789],{"className":4788},[542,543,544,545],[47,4790,3366],{"className":4791,"style":3580},[508,581,545],[47,4793],{"className":4794,"style":889},[888],[47,4796,852],{"className":4797},[893],[47,4799],{"className":4800,"style":889},[888],[47,4802,4804,4807],{"className":4803},[499],[47,4805],{"className":4806,"style":903},[503],[47,4808,4810,4813],{"className":4809},[508],[47,4811,859],{"className":4812},[508,910],[47,4814,4816],{"className":4815},[585],[47,4817,4819],{"className":4818},[520],[47,4820,4822],{"className":4821},[525],[47,4823,4825],{"className":4824,"style":903},[529],[47,4826,4827,4830],{"style":925},[47,4828],{"className":4829,"style":929},[537],[47,4831,4833],{"className":4832},[542,543,544,545],[47,4834,4836,4839,4879,4882],{"className":4835},[508,545],[47,4837,3327],{"className":4838},[508,581,545],[47,4840,4842,4845],{"className":4841},[508,545],[47,4843,482],{"className":4844},[508,581,545],[47,4846,4848],{"className":4847},[585],[47,4849,4851,4871],{"className":4850},[520,521],[47,4852,4854,4868],{"className":4853},[525],[47,4855,4857],{"className":4856,"style":4672},[529],[47,4858,4859,4862],{"style":4675},[47,4860],{"className":4861,"style":602},[537],[47,4863,4865],{"className":4864},[542,606,607,545],[47,4866,815],{"className":4867,"style":830},[508,581,545],[47,4869,616],{"className":4870},[615],[47,4872,4874],{"className":4873},[525],[47,4875,4877],{"className":4876,"style":4694},[529],[47,4878],{},[47,4880,866],{"className":4881},[942,545],[47,4883,4885,4888],{"className":4884},[508,545],[47,4886,482],{"className":4887},[508,581,545],[47,4889,4891],{"className":4890},[585],[47,4892,4894,4920],{"className":4893},[520,521],[47,4895,4897,4917],{"className":4896},[525],[47,4898,4900],{"className":4899,"style":595},[529],[47,4901,4902,4905],{"style":598},[47,4903],{"className":4904,"style":602},[537],[47,4906,4908],{"className":4907},[542,606,607,545],[47,4909,4911],{"className":4910},[508,545],[47,4912,4914],{"className":4913},[508,545],[47,4915,4141],{"className":4916},[508,1342,545],[47,4918,616],{"className":4919},[615],[47,4921,4923],{"className":4922},[525],[47,4924,4926],{"className":4925,"style":623},[529],[47,4927],{},[14,4929,4930,4931,4934,4935,4938,4939,4941,4942,4944],{},"为了能使多个头并行计算，我们定义两个转置函数 ",[44,4932,4933],{},"transpose_output"," 和 ",[44,4936,4937],{},"transpose_qkv","，其中 ",[44,4940,4933],{}," 还原了 ",[44,4943,4937],{}," 的结果。",[37,4946,4948],{"className":2778,"code":4947,"language":2780,"meta":42,"style":42},"def transpose_qkv(X, num_heads):\n    \"\"\"为了多注意力头的并行计算而变换形状\"\"\"\n    # 输入X的形状:(batch_size，查询或者“键－值”对的个数，num_hiddens)\n    # 输出X的形状:(batch_size，查询或者“键－值”对的个数，num_heads，num_hiddens/num_heads)\n    X = X.reshape(X.shape[0], X.shape[1], num_heads, -1)\n\n    # 输出X的形状:(batch_size，num_heads，查询或者“键－值”对的个数, num_hiddens/num_heads)\n    X = X.permute(0, 2, 1, 3)\n\n    # 最终输出的形状:(batch_size*num_heads,查询或者“键－值”对的个数, num_hiddens/num_heads)\n    return X.reshape(-1, X.shape[2], X.shape[3])\n\ndef transpose_output(X, num_heads):\n    \"\"\"逆转transpose_qkv函数的操作\"\"\"\n    X = X.reshape(-1, num_heads, X.shape[1], X.shape[2])\n    X = X.permute(0, 2, 1, 3)\n    return X.reshape(X.shape[0], X.shape[1], -1)\n",[44,4949,4950,4968,4973,4978,4983,5009,5013,5018,5044,5048,5053,5076,5080,5097,5102,5125,5149],{"__ignoreMap":42},[47,4951,4952,4954,4957,4959,4961,4963,4966],{"class":49,"line":50},[47,4953,2787],{"class":81},[47,4955,4956],{"class":53}," transpose_qkv",[47,4958,1187],{"class":353},[47,4960,2796],{"class":2795},[47,4962,2799],{"class":353},[47,4964,4965],{"class":2795},"num_heads",[47,4967,2805],{"class":353},[47,4969,4970],{"class":49,"line":64},[47,4971,4972],{"class":2810},"    \"\"\"为了多注意力头的并行计算而变换形状\"\"\"\n",[47,4974,4975],{"class":49,"line":74},[47,4976,4977],{"class":331},"    # 输入X的形状:(batch_size，查询或者“键－值”对的个数，num_hiddens)\n",[47,4979,4980],{"class":49,"line":335},[47,4981,4982],{"class":331},"    # 输出X的形状:(batch_size，查询或者“键－值”对的个数，num_heads，num_hiddens/num_heads)\n",[47,4984,4985,4988,4990,4993,4995,4998,5000,5003,5005,5007],{"class":49,"line":344},[47,4986,4987],{"class":353},"    X ",[47,4989,82],{"class":81},[47,4991,4992],{"class":353}," X.reshape(X.shape[",[47,4994,2141],{"class":118},[47,4996,4997],{"class":353},"], X.shape[",[47,4999,472],{"class":118},[47,5001,5002],{"class":353},"], num_heads, ",[47,5004,2926],{"class":81},[47,5006,472],{"class":118},[47,5008,2853],{"class":353},[47,5010,5011],{"class":49,"line":2856},[47,5012,326],{"emptyLinePlaceholder":325},[47,5014,5015],{"class":49,"line":2864},[47,5016,5017],{"class":331},"    # 输出X的形状:(batch_size，num_heads，查询或者“键－值”对的个数, num_hiddens/num_heads)\n",[47,5019,5020,5022,5024,5027,5029,5031,5033,5035,5037,5039,5042],{"class":49,"line":2875},[47,5021,4987],{"class":353},[47,5023,82],{"class":81},[47,5025,5026],{"class":353}," X.permute(",[47,5028,2141],{"class":118},[47,5030,2799],{"class":353},[47,5032,3192],{"class":118},[47,5034,2799],{"class":353},[47,5036,472],{"class":118},[47,5038,2799],{"class":353},[47,5040,5041],{"class":118},"3",[47,5043,2853],{"class":353},[47,5045,5046],{"class":49,"line":2892},[47,5047,326],{"emptyLinePlaceholder":325},[47,5049,5050],{"class":49,"line":2908},[47,5051,5052],{"class":331},"    # 最终输出的形状:(batch_size*num_heads,查询或者“键－值”对的个数, num_hiddens/num_heads)\n",[47,5054,5055,5058,5061,5063,5065,5068,5070,5072,5074],{"class":49,"line":2916},[47,5056,5057],{"class":81},"    return",[47,5059,5060],{"class":353}," X.reshape(",[47,5062,2926],{"class":81},[47,5064,472],{"class":118},[47,5066,5067],{"class":353},", X.shape[",[47,5069,3192],{"class":118},[47,5071,4997],{"class":353},[47,5073,5041],{"class":118},[47,5075,2905],{"class":353},[47,5077,5078],{"class":49,"line":2933},[47,5079,326],{"emptyLinePlaceholder":325},[47,5081,5082,5084,5087,5089,5091,5093,5095],{"class":49,"line":2939},[47,5083,2787],{"class":81},[47,5085,5086],{"class":53}," transpose_output",[47,5088,1187],{"class":353},[47,5090,2796],{"class":2795},[47,5092,2799],{"class":353},[47,5094,4965],{"class":2795},[47,5096,2805],{"class":353},[47,5098,5099],{"class":49,"line":2964},[47,5100,5101],{"class":2810},"    \"\"\"逆转transpose_qkv函数的操作\"\"\"\n",[47,5103,5104,5106,5108,5110,5112,5114,5117,5119,5121,5123],{"class":49,"line":2977},[47,5105,4987],{"class":353},[47,5107,82],{"class":81},[47,5109,5060],{"class":353},[47,5111,2926],{"class":81},[47,5113,472],{"class":118},[47,5115,5116],{"class":353},", num_heads, X.shape[",[47,5118,472],{"class":118},[47,5120,4997],{"class":353},[47,5122,3192],{"class":118},[47,5124,2905],{"class":353},[47,5126,5127,5129,5131,5133,5135,5137,5139,5141,5143,5145,5147],{"class":49,"line":3215},[47,5128,4987],{"class":353},[47,5130,82],{"class":81},[47,5132,5026],{"class":353},[47,5134,2141],{"class":118},[47,5136,2799],{"class":353},[47,5138,3192],{"class":118},[47,5140,2799],{"class":353},[47,5142,472],{"class":118},[47,5144,2799],{"class":353},[47,5146,5041],{"class":118},[47,5148,2853],{"class":353},[47,5150,5152,5154,5156,5158,5160,5162,5165,5167,5169],{"class":49,"line":5151},17,[47,5153,5057],{"class":81},[47,5155,4992],{"class":353},[47,5157,2141],{"class":118},[47,5159,4997],{"class":353},[47,5161,472],{"class":118},[47,5163,5164],{"class":353},"], ",[47,5166,2926],{"class":81},[47,5168,472],{"class":118},[47,5170,2853],{"class":353},[37,5172,5174],{"className":2778,"code":5173,"language":2780,"meta":42,"style":42},"#@save\nclass MultiHeadAttention(nn.Module):\n    \"\"\"多头注意力\"\"\"\n    def __init__(self, key_size, query_size, value_size, num_hiddens,\n                 num_heads, dropout, bias=False, **kwargs):\n        super(MultiHeadAttention, self).__init__(**kwargs)\n        self.num_heads = num_heads\n        self.attention = d2l.DotProductAttention(dropout)\n        self.W_q = nn.Linear(query_size, num_hiddens, bias=bias)\n        self.W_k = nn.Linear(key_size, num_hiddens, bias=bias)\n        self.W_v = nn.Linear(value_size, num_hiddens, bias=bias)\n        self.W_o = nn.Linear(num_hiddens, num_hiddens, bias=bias)\n\n    def forward(self, queries, keys, values, valid_lens):\n        # queries，keys，values的形状:\n        # (batch_size，查询或者“键－值”对的个数，num_hiddens)\n        # valid_lens　的形状:\n        # (batch_size，)或(batch_size，查询的个数)\n        # 经过变换后，输出的queries，keys，values 的形状:\n        # (batch_size*num_heads，查询或者“键－值”对的个数，num_hiddens/num_heads)\n        queries = transpose_qkv(self.W_q(queries), self.num_heads)\n        keys = transpose_qkv(self.W_k(keys), self.num_heads)\n        values = transpose_qkv(self.W_v(values), self.num_heads)\n\n        if valid_lens is not None:\n            # 在轴0，将第一项（标量或者矢量）复制num_heads次，\n            # 然后如此复制第二项，然后诸如此类。\n            valid_lens = torch.repeat_interleave(\n                valid_lens, repeats=self.num_heads, dim=0)\n\n        # output的形状:(batch_size*num_heads，查询的个数， num_hiddens/num_heads)\n        output = self.attention(queries, keys, values, valid_lens)\n\n        # output_concat的形状:(batch_size，查询的个数，num_hiddens)\n        output_concat = transpose_output(output, self.num_heads)\n        return self.W_o(output_concat)\n",[44,5175,5176,5181,5198,5203,5235,5262,5281,5293,5305,5324,5342,5360,5378,5382,5410,5415,5420,5425,5431,5437,5443,5464,5483,5502,5507,5523,5529,5535,5545,5569,5574,5580,5594,5599,5605,5620],{"__ignoreMap":42},[47,5177,5178],{"class":49,"line":50},[47,5179,5180],{"class":331},"#@save\n",[47,5182,5183,5185,5188,5190,5192,5194,5196],{"class":49,"line":64},[47,5184,3003],{"class":81},[47,5186,5187],{"class":3006}," MultiHeadAttention",[47,5189,1187],{"class":353},[47,5191,3013],{"class":3012},[47,5193,144],{"class":353},[47,5195,3018],{"class":3012},[47,5197,2805],{"class":353},[47,5199,5200],{"class":49,"line":74},[47,5201,5202],{"class":2810},"    \"\"\"多头注意力\"\"\"\n",[47,5204,5205,5207,5209,5211,5213,5215,5218,5220,5223,5225,5228,5230,5233],{"class":49,"line":335},[47,5206,3030],{"class":81},[47,5208,3033],{"class":118},[47,5210,1187],{"class":353},[47,5212,3039],{"class":3038},[47,5214,2799],{"class":353},[47,5216,5217],{"class":2795},"key_size",[47,5219,2799],{"class":353},[47,5221,5222],{"class":2795},"query_size",[47,5224,2799],{"class":353},[47,5226,5227],{"class":2795},"value_size",[47,5229,2799],{"class":353},[47,5231,5232],{"class":2795},"num_hiddens",[47,5234,4196],{"class":353},[47,5236,5237,5240,5242,5244,5246,5249,5251,5254,5256,5258,5260],{"class":49,"line":344},[47,5238,5239],{"class":2795},"                 num_heads",[47,5241,2799],{"class":353},[47,5243,3044],{"class":2795},[47,5245,2799],{"class":353},[47,5247,5248],{"class":2795},"bias",[47,5250,82],{"class":81},[47,5252,5253],{"class":118},"False",[47,5255,2799],{"class":353},[47,5257,3049],{"class":81},[47,5259,3052],{"class":2795},[47,5261,2805],{"class":353},[47,5263,5264,5266,5269,5271,5273,5275,5277,5279],{"class":49,"line":2856},[47,5265,3060],{"class":3059},[47,5267,5268],{"class":353},"(MultiHeadAttention, ",[47,5270,3039],{"class":3066},[47,5272,3069],{"class":353},[47,5274,3072],{"class":118},[47,5276,1187],{"class":353},[47,5278,3049],{"class":81},[47,5280,3079],{"class":353},[47,5282,5283,5285,5288,5290],{"class":49,"line":2864},[47,5284,3084],{"class":3066},[47,5286,5287],{"class":353},".num_heads ",[47,5289,82],{"class":81},[47,5291,5292],{"class":353}," num_heads\n",[47,5294,5295,5297,5300,5302],{"class":49,"line":2875},[47,5296,3084],{"class":3066},[47,5298,5299],{"class":353},".attention ",[47,5301,82],{"class":81},[47,5303,5304],{"class":353}," d2l.DotProductAttention(dropout)\n",[47,5306,5307,5309,5312,5314,5317,5319,5321],{"class":49,"line":2892},[47,5308,3084],{"class":3066},[47,5310,5311],{"class":353},".W_q ",[47,5313,82],{"class":81},[47,5315,5316],{"class":353}," nn.Linear(query_size, num_hiddens, ",[47,5318,5248],{"class":2844},[47,5320,82],{"class":81},[47,5322,5323],{"class":353},"bias)\n",[47,5325,5326,5328,5331,5333,5336,5338,5340],{"class":49,"line":2908},[47,5327,3084],{"class":3066},[47,5329,5330],{"class":353},".W_k ",[47,5332,82],{"class":81},[47,5334,5335],{"class":353}," nn.Linear(key_size, num_hiddens, ",[47,5337,5248],{"class":2844},[47,5339,82],{"class":81},[47,5341,5323],{"class":353},[47,5343,5344,5346,5349,5351,5354,5356,5358],{"class":49,"line":2916},[47,5345,3084],{"class":3066},[47,5347,5348],{"class":353},".W_v ",[47,5350,82],{"class":81},[47,5352,5353],{"class":353}," nn.Linear(value_size, num_hiddens, ",[47,5355,5248],{"class":2844},[47,5357,82],{"class":81},[47,5359,5323],{"class":353},[47,5361,5362,5364,5367,5369,5372,5374,5376],{"class":49,"line":2933},[47,5363,3084],{"class":3066},[47,5365,5366],{"class":353},".W_o ",[47,5368,82],{"class":81},[47,5370,5371],{"class":353}," nn.Linear(num_hiddens, num_hiddens, ",[47,5373,5248],{"class":2844},[47,5375,82],{"class":81},[47,5377,5323],{"class":353},[47,5379,5380],{"class":49,"line":2939},[47,5381,326],{"emptyLinePlaceholder":325},[47,5383,5384,5386,5388,5390,5392,5394,5396,5398,5400,5402,5404,5406,5408],{"class":49,"line":2964},[47,5385,3030],{"class":81},[47,5387,3123],{"class":53},[47,5389,1187],{"class":353},[47,5391,3039],{"class":3038},[47,5393,2799],{"class":353},[47,5395,3132],{"class":2795},[47,5397,2799],{"class":353},[47,5399,3137],{"class":2795},[47,5401,2799],{"class":353},[47,5403,3142],{"class":2795},[47,5405,2799],{"class":353},[47,5407,2802],{"class":2795},[47,5409,2805],{"class":353},[47,5411,5412],{"class":49,"line":2977},[47,5413,5414],{"class":331},"        # queries，keys，values的形状:\n",[47,5416,5417],{"class":49,"line":3215},[47,5418,5419],{"class":331},"        # (batch_size，查询或者“键－值”对的个数，num_hiddens)\n",[47,5421,5422],{"class":49,"line":5151},[47,5423,5424],{"class":331},"        # valid_lens　的形状:\n",[47,5426,5428],{"class":49,"line":5427},18,[47,5429,5430],{"class":331},"        # (batch_size，)或(batch_size，查询的个数)\n",[47,5432,5434],{"class":49,"line":5433},19,[47,5435,5436],{"class":331},"        # 经过变换后，输出的queries，keys，values 的形状:\n",[47,5438,5440],{"class":49,"line":5439},20,[47,5441,5442],{"class":331},"        # (batch_size*num_heads，查询或者“键－值”对的个数，num_hiddens/num_heads)\n",[47,5444,5446,5449,5451,5454,5456,5459,5461],{"class":49,"line":5445},21,[47,5447,5448],{"class":353},"        queries ",[47,5450,82],{"class":81},[47,5452,5453],{"class":353}," transpose_qkv(",[47,5455,3039],{"class":3066},[47,5457,5458],{"class":353},".W_q(queries), ",[47,5460,3039],{"class":3066},[47,5462,5463],{"class":353},".num_heads)\n",[47,5465,5467,5470,5472,5474,5476,5479,5481],{"class":49,"line":5466},22,[47,5468,5469],{"class":353},"        keys ",[47,5471,82],{"class":81},[47,5473,5453],{"class":353},[47,5475,3039],{"class":3066},[47,5477,5478],{"class":353},".W_k(keys), ",[47,5480,3039],{"class":3066},[47,5482,5463],{"class":353},[47,5484,5486,5489,5491,5493,5495,5498,5500],{"class":49,"line":5485},23,[47,5487,5488],{"class":353},"        values ",[47,5490,82],{"class":81},[47,5492,5453],{"class":353},[47,5494,3039],{"class":3066},[47,5496,5497],{"class":353},".W_v(values), ",[47,5499,3039],{"class":3066},[47,5501,5463],{"class":353},[47,5503,5505],{"class":49,"line":5504},24,[47,5506,326],{"emptyLinePlaceholder":325},[47,5508,5510,5512,5514,5516,5519,5521],{"class":49,"line":5509},25,[47,5511,2878],{"class":81},[47,5513,2824],{"class":353},[47,5515,2827],{"class":81},[47,5517,5518],{"class":81}," not",[47,5520,2830],{"class":118},[47,5522,2833],{"class":353},[47,5524,5526],{"class":49,"line":5525},26,[47,5527,5528],{"class":331},"            # 在轴0，将第一项（标量或者矢量）复制num_heads次，\n",[47,5530,5532],{"class":49,"line":5531},27,[47,5533,5534],{"class":331},"            # 然后如此复制第二项，然后诸如此类。\n",[47,5536,5538,5540,5542],{"class":49,"line":5537},28,[47,5539,2895],{"class":353},[47,5541,82],{"class":81},[47,5543,5544],{"class":353}," torch.repeat_interleave(\n",[47,5546,5548,5551,5554,5556,5558,5561,5563,5565,5567],{"class":49,"line":5547},29,[47,5549,5550],{"class":353},"                valid_lens, ",[47,5552,5553],{"class":2844},"repeats",[47,5555,82],{"class":81},[47,5557,3039],{"class":3066},[47,5559,5560],{"class":353},".num_heads, ",[47,5562,2845],{"class":2844},[47,5564,82],{"class":81},[47,5566,2141],{"class":118},[47,5568,2853],{"class":353},[47,5570,5572],{"class":49,"line":5571},30,[47,5573,326],{"emptyLinePlaceholder":325},[47,5575,5577],{"class":49,"line":5576},31,[47,5578,5579],{"class":331},"        # output的形状:(batch_size*num_heads，查询的个数， num_hiddens/num_heads)\n",[47,5581,5583,5586,5588,5591],{"class":49,"line":5582},32,[47,5584,5585],{"class":353},"        output ",[47,5587,82],{"class":81},[47,5589,5590],{"class":3066}," self",[47,5592,5593],{"class":353},".attention(queries, keys, values, valid_lens)\n",[47,5595,5597],{"class":49,"line":5596},33,[47,5598,326],{"emptyLinePlaceholder":325},[47,5600,5602],{"class":49,"line":5601},34,[47,5603,5604],{"class":331},"        # output_concat的形状:(batch_size，查询的个数，num_hiddens)\n",[47,5606,5608,5611,5613,5616,5618],{"class":49,"line":5607},35,[47,5609,5610],{"class":353},"        output_concat ",[47,5612,82],{"class":81},[47,5614,5615],{"class":353}," transpose_output(output, ",[47,5617,3039],{"class":3066},[47,5619,5463],{"class":353},[47,5621,5623,5625,5627],{"class":49,"line":5622},36,[47,5624,2838],{"class":81},[47,5626,5590],{"class":3066},[47,5628,5629],{"class":353},".W_o(output_concat)\n",[10,5631,5632],{"id":5632},"位置编码",[14,5634,5635,5636,5727],{},"在处理 token 序列时 RNN 采用了按顺序逐个处理 token 的，但注意力机制为了并行计算放弃了顺序处理，这会导致其失去位置信息。为了使模型能够使用到位置信息，我们需要在 token 中注入一些和位置相关的信息，我们使用位置编码来注入绝对的或相对的位置信息。位置编码与嵌入的维度相同，均为 ",[47,5637,5639,5665],{"className":5638},[451],[47,5640,5642],{"className":5641},[455],[457,5643,5644],{"xmlns":459},[461,5645,5646,5662],{},[464,5647,5648,5654,5656,5658,5660],{},[477,5649,5650,5652],{},[480,5651,482],{},[480,5653,755],{},[480,5655,1181],{},[480,5657,482],{},[480,5659,1171],{},[480,5661,3273],{},[487,5663,5664],{"encoding":489},"d_model",[47,5666,5668],{"className":5667,"ariaHidden":495},[494],[47,5669,5671,5674,5714,5717,5720,5723],{"className":5670},[499],[47,5672],{"className":5673,"style":1616},[503],[47,5675,5677,5680],{"className":5676},[508],[47,5678,482],{"className":5679},[508,581],[47,5681,5683],{"className":5682},[585],[47,5684,5686,5706],{"className":5685},[520,521],[47,5687,5689,5703],{"className":5688},[525],[47,5690,5692],{"className":5691,"style":1709},[529],[47,5693,5694,5697],{"style":1638},[47,5695],{"className":5696,"style":929},[537],[47,5698,5700],{"className":5699},[542,543,544,545],[47,5701,755],{"className":5702},[508,581,545],[47,5704,616],{"className":5705},[615],[47,5707,5709],{"className":5708},[525],[47,5710,5712],{"className":5711,"style":1657},[529],[47,5713],{},[47,5715,1181],{"className":5716},[508,581],[47,5718,482],{"className":5719},[508,581],[47,5721,1171],{"className":5722},[508,581],[47,5724,3273],{"className":5725,"style":5726},[508,581],"margin-right:0.01968em;","，因此两者可以相加。位置编码可以通过学习得到也可以直接固定得到。",[14,5729,5730],{},"在这里，我们使用的是基于正弦函数和余弦函数的位置编码。",[14,5732,5733,5734,5899,5900,5928,5929,5957,5958,6130,6131,6192,6193,6222,6223,6260,6261,6317],{},"假设输入表示 ",[47,5735,5737,5780],{"className":5736},[451],[47,5738,5740],{"className":5739},[455],[457,5741,5742],{"xmlns":459},[461,5743,5744,5777],{},[464,5745,5746,5749,5751],{},[480,5747,2796],{"mathvariant":5748},"bold",[850,5750,852],{},[854,5752,5753,5755],{},[480,5754,859],{"mathvariant":858},[464,5756,5757,5759,5761],{},[480,5758,724],{},[850,5760,866],{},[477,5762,5763,5765],{},[480,5764,482],{},[464,5766,5767,5769,5771,5773,5775],{},[480,5768,755],{},[480,5770,1181],{},[480,5772,482],{},[480,5774,1171],{},[480,5776,3273],{},[487,5778,5779],{"encoding":489},"\\mathbf{X} \\in \\mathbb{R}^{n \\times d_{model}}",[47,5781,5783,5803],{"className":5782,"ariaHidden":495},[494],[47,5784,5786,5790,5794,5797,5800],{"className":5785},[499],[47,5787],{"className":5788,"style":5789},[503],"height:0.7252em;vertical-align:-0.0391em;",[47,5791,2796],{"className":5792},[508,5793],"mathbf",[47,5795],{"className":5796,"style":889},[888],[47,5798,852],{"className":5799},[893],[47,5801],{"className":5802,"style":889},[888],[47,5804,5806,5809],{"className":5805},[499],[47,5807],{"className":5808,"style":903},[503],[47,5810,5812,5815],{"className":5811},[508],[47,5813,859],{"className":5814},[508,910],[47,5816,5818],{"className":5817},[585],[47,5819,5821],{"className":5820},[520],[47,5822,5824],{"className":5823},[525],[47,5825,5827],{"className":5826,"style":903},[529],[47,5828,5829,5832],{"style":925},[47,5830],{"className":5831,"style":929},[537],[47,5833,5835],{"className":5834},[542,543,544,545],[47,5836,5838,5841,5844],{"className":5837},[508,545],[47,5839,724],{"className":5840},[508,581,545],[47,5842,866],{"className":5843},[942,545],[47,5845,5847,5850],{"className":5846},[508,545],[47,5848,482],{"className":5849},[508,581,545],[47,5851,5853],{"className":5852},[585],[47,5854,5856,5891],{"className":5855},[520,521],[47,5857,5859,5888],{"className":5858},[525],[47,5860,5862],{"className":5861,"style":595},[529],[47,5863,5864,5867],{"style":598},[47,5865],{"className":5866,"style":602},[537],[47,5868,5870],{"className":5869},[542,606,607,545],[47,5871,5873,5876,5879,5882,5885],{"className":5872},[508,545],[47,5874,755],{"className":5875},[508,581,545],[47,5877,1181],{"className":5878},[508,581,545],[47,5880,482],{"className":5881},[508,581,545],[47,5883,1171],{"className":5884},[508,581,545],[47,5886,3273],{"className":5887,"style":5726},[508,581,545],[47,5889,616],{"className":5890},[615],[47,5892,5894],{"className":5893},[525],[47,5895,5897],{"className":5896,"style":623},[529],[47,5898],{}," 包含一个序列中 ",[47,5901,5903,5916],{"className":5902},[451],[47,5904,5906],{"className":5905},[455],[457,5907,5908],{"xmlns":459},[461,5909,5910,5914],{},[464,5911,5912],{},[480,5913,724],{},[487,5915,724],{"encoding":489},[47,5917,5919],{"className":5918,"ariaHidden":495},[494],[47,5920,5922,5925],{"className":5921},[499],[47,5923],{"className":5924,"style":736},[503],[47,5926,724],{"className":5927},[508,581]," 个词元的 ",[47,5930,5932,5945],{"className":5931},[451],[47,5933,5935],{"className":5934},[455],[457,5936,5937],{"xmlns":459},[461,5938,5939,5943],{},[464,5940,5941],{},[480,5942,482],{},[487,5944,482],{"encoding":489},[47,5946,5948],{"className":5947,"ariaHidden":495},[494],[47,5949,5951,5954],{"className":5950},[499],[47,5952],{"className":5953,"style":796},[503],[47,5955,482],{"className":5956},[508,581]," 维嵌入表示。位置编码使用相同形状的位置嵌入矩阵 ",[47,5959,5961,6009],{"className":5960},[451],[47,5962,5964],{"className":5963},[455],[457,5965,5966],{"xmlns":459},[461,5967,5968,6006],{},[464,5969,5970,5978,5980],{},[464,5971,5972,5975],{},[480,5973,5974],{"mathvariant":5748},"P",[480,5976,5977],{"mathvariant":5748},"E",[850,5979,852],{},[854,5981,5982,5984],{},[480,5983,859],{"mathvariant":858},[464,5985,5986,5988,5990],{},[480,5987,724],{},[850,5989,866],{},[477,5991,5992,5994],{},[480,5993,482],{},[464,5995,5996,5998,6000,6002,6004],{},[480,5997,755],{},[480,5999,1181],{},[480,6001,482],{},[480,6003,1171],{},[480,6005,3273],{},[487,6007,6008],{"encoding":489},"\\mathbf{PE} \\in \\mathbb{R}^{n \\times d_{model}}",[47,6010,6012,6034],{"className":6011,"ariaHidden":495},[494],[47,6013,6015,6018,6025,6028,6031],{"className":6014},[499],[47,6016],{"className":6017,"style":5789},[503],[47,6019,6021],{"className":6020},[508],[47,6022,6024],{"className":6023},[508,5793],"PE",[47,6026],{"className":6027,"style":889},[888],[47,6029,852],{"className":6030},[893],[47,6032],{"className":6033,"style":889},[888],[47,6035,6037,6040],{"className":6036},[499],[47,6038],{"className":6039,"style":903},[503],[47,6041,6043,6046],{"className":6042},[508],[47,6044,859],{"className":6045},[508,910],[47,6047,6049],{"className":6048},[585],[47,6050,6052],{"className":6051},[520],[47,6053,6055],{"className":6054},[525],[47,6056,6058],{"className":6057,"style":903},[529],[47,6059,6060,6063],{"style":925},[47,6061],{"className":6062,"style":929},[537],[47,6064,6066],{"className":6065},[542,543,544,545],[47,6067,6069,6072,6075],{"className":6068},[508,545],[47,6070,724],{"className":6071},[508,581,545],[47,6073,866],{"className":6074},[942,545],[47,6076,6078,6081],{"className":6077},[508,545],[47,6079,482],{"className":6080},[508,581,545],[47,6082,6084],{"className":6083},[585],[47,6085,6087,6122],{"className":6086},[520,521],[47,6088,6090,6119],{"className":6089},[525],[47,6091,6093],{"className":6092,"style":595},[529],[47,6094,6095,6098],{"style":598},[47,6096],{"className":6097,"style":602},[537],[47,6099,6101],{"className":6100},[542,606,607,545],[47,6102,6104,6107,6110,6113,6116],{"className":6103},[508,545],[47,6105,755],{"className":6106},[508,581,545],[47,6108,1181],{"className":6109},[508,581,545],[47,6111,482],{"className":6112},[508,581,545],[47,6114,1171],{"className":6115},[508,581,545],[47,6117,3273],{"className":6118,"style":5726},[508,581,545],[47,6120,616],{"className":6121},[615],[47,6123,6125],{"className":6124},[525],[47,6126,6128],{"className":6127,"style":623},[529],[47,6129],{}," 输出 ",[47,6132,6134,6157],{"className":6133},[451],[47,6135,6137],{"className":6136},[455],[457,6138,6139],{"xmlns":459},[461,6140,6141,6154],{},[464,6142,6143,6145,6148],{},[480,6144,2796],{"mathvariant":5748},[850,6146,6147],{},"+",[464,6149,6150,6152],{},[480,6151,5974],{"mathvariant":5748},[480,6153,5977],{"mathvariant":5748},[487,6155,6156],{"encoding":489},"\\mathbf{X} + \\mathbf{PE}",[47,6158,6160,6179],{"className":6159,"ariaHidden":495},[494],[47,6161,6163,6167,6170,6173,6176],{"className":6162},[499],[47,6164],{"className":6165,"style":6166},[503],"height:0.7694em;vertical-align:-0.0833em;",[47,6168,2796],{"className":6169},[508,5793],[47,6171],{"className":6172,"style":2259},[888],[47,6174,6147],{"className":6175},[942],[47,6177],{"className":6178,"style":2259},[888],[47,6180,6182,6186],{"className":6181},[499],[47,6183],{"className":6184,"style":6185},[503],"height:0.6861em;",[47,6187,6189],{"className":6188},[508],[47,6190,6024],{"className":6191},[508,5793],"，矩阵第 ",[47,6194,6196,6209],{"className":6195},[451],[47,6197,6199],{"className":6198},[455],[457,6200,6201],{"xmlns":459},[461,6202,6203,6207],{},[464,6204,6205],{},[480,6206,1178],{},[487,6208,1178],{"encoding":489},[47,6210,6212],{"className":6211,"ariaHidden":495},[494],[47,6213,6215,6219],{"className":6214},[499],[47,6216],{"className":6217,"style":6218},[503],"height:0.6595em;",[47,6220,1178],{"className":6221},[508,581]," 行、第 ",[47,6224,6226,6243],{"className":6225},[451],[47,6227,6229],{"className":6228},[455],[457,6230,6231],{"xmlns":459},[461,6232,6233,6240],{},[464,6234,6235,6237],{},[470,6236,3192],{},[480,6238,6239],{},"j",[487,6241,6242],{"encoding":489},"2j",[47,6244,6246],{"className":6245,"ariaHidden":495},[494],[47,6247,6249,6253,6256],{"className":6248},[499],[47,6250],{"className":6251,"style":6252},[503],"height:0.854em;vertical-align:-0.1944em;",[47,6254,3192],{"className":6255},[508],[47,6257,6239],{"className":6258,"style":6259},[508,581],"margin-right:0.05724em;"," 列和 ",[47,6262,6264,6284],{"className":6263},[451],[47,6265,6267],{"className":6266},[455],[457,6268,6269],{"xmlns":459},[461,6270,6271,6281],{},[464,6272,6273,6275,6277,6279],{},[470,6274,3192],{},[480,6276,6239],{},[850,6278,6147],{},[470,6280,472],{},[487,6282,6283],{"encoding":489},"2j+1",[47,6285,6287,6308],{"className":6286,"ariaHidden":495},[494],[47,6288,6290,6293,6296,6299,6302,6305],{"className":6289},[499],[47,6291],{"className":6292,"style":6252},[503],[47,6294,3192],{"className":6295},[508],[47,6297,6239],{"className":6298,"style":6259},[508,581],[47,6300],{"className":6301,"style":2259},[888],[47,6303,6147],{"className":6304},[942],[47,6306],{"className":6307,"style":2259},[888],[47,6309,6311,6314],{"className":6310},[499],[47,6312],{"className":6313,"style":2153},[503],[47,6315,472],{"className":6316},[508]," 列上的元素为：",[14,6319,6320],{},[47,6321,6323,6401],{"className":6322},[451],[47,6324,6326],{"className":6325},[455],[457,6327,6328],{"xmlns":459},[461,6329,6330,6398],{},[464,6331,6332,6334,6352,6354,6357,6359],{},[480,6333,5974],{},[477,6335,6336,6338],{},[480,6337,5977],{},[464,6339,6340,6342,6344,6346,6348,6350],{},[850,6341,1187],{"stretchy":1186},[480,6343,1178],{},[850,6345,1192],{"separator":495},[470,6347,3192],{},[480,6349,6239],{},[850,6351,1201],{"stretchy":1186},[850,6353,82],{},[480,6355,6356],{},"sin",[850,6358,1210],{},[464,6360,6361,6363,6396],{},[850,6362,1187],{"fence":495},[467,6364,6365,6367],{},[480,6366,1178],{},[854,6368,6369,6372],{},[470,6370,6371],{},"10000",[464,6373,6374,6376,6378,6380],{},[470,6375,3192],{},[480,6377,6239],{},[480,6379,141],{"mathvariant":1206},[477,6381,6382,6384],{},[480,6383,482],{},[464,6385,6386,6388,6390,6392,6394],{},[480,6387,755],{},[480,6389,1181],{},[480,6391,482],{},[480,6393,1171],{},[480,6395,3273],{},[850,6397,1201],{"fence":495},[487,6399,6400],{"encoding":489}," PE_{(i,2j)} = \\sin\\left(\\frac{i}{10000^{2j/d_{model}}}\\right)",[47,6402,6404,6484],{"className":6403,"ariaHidden":495},[494],[47,6405,6407,6411,6414,6475,6478,6481],{"className":6406},[499],[47,6408],{"className":6409,"style":6410},[503],"height:1.0385em;vertical-align:-0.3552em;",[47,6412,5974],{"className":6413,"style":3555},[508,581],[47,6415,6417,6421],{"className":6416},[508],[47,6418,5977],{"className":6419,"style":6420},[508,581],"margin-right:0.05764em;",[47,6422,6424],{"className":6423},[585],[47,6425,6427,6466],{"className":6426},[520,521],[47,6428,6430,6463],{"className":6429},[525],[47,6431,6433],{"className":6432,"style":595},[529],[47,6434,6436,6439],{"style":6435},"top:-2.5198em;margin-left:-0.0576em;margin-right:0.05em;",[47,6437],{"className":6438,"style":929},[537],[47,6440,6442],{"className":6441},[542,543,544,545],[47,6443,6445,6448,6451,6454,6457,6460],{"className":6444},[508,545],[47,6446,1187],{"className":6447},[512,545],[47,6449,1178],{"className":6450},[508,581,545],[47,6452,1192],{"className":6453},[1299,545],[47,6455,3192],{"className":6456},[508,545],[47,6458,6239],{"className":6459,"style":6259},[508,581,545],[47,6461,1201],{"className":6462},[702,545],[47,6464,616],{"className":6465},[615],[47,6467,6469],{"className":6468},[525],[47,6470,6473],{"className":6471,"style":6472},[529],"height:0.3552em;",[47,6474],{},[47,6476],{"className":6477,"style":889},[888],[47,6479,82],{"className":6480},[893],[47,6482],{"className":6483,"style":889},[888],[47,6485,6487,6490,6493,6496],{"className":6486},[499],[47,6488],{"className":6489,"style":1334},[503],[47,6491,6356],{"className":6492},[1338],[47,6494],{"className":6495,"style":1303},[888],[47,6497,6499,6505,6673],{"className":6498},[1349],[47,6500,6502],{"className":6501,"style":1354},[512,1353],[47,6503,1187],{"className":6504},[1358,1359],[47,6506,6508,6511,6670],{"className":6507},[508],[47,6509],{"className":6510},[512,513],[47,6512,6514],{"className":6513},[467],[47,6515,6517,6661],{"className":6516},[520,521],[47,6518,6520,6658],{"className":6519},[525],[47,6521,6524,6636,6644],{"className":6522,"style":6523},[529],"height:0.8557em;",[47,6525,6527,6530],{"style":6526},"top:-2.5648em;",[47,6528],{"className":6529,"style":538},[537],[47,6531,6533],{"className":6532},[542,543,544,545],[47,6534,6536,6540],{"className":6535},[508,545],[47,6537,6539],{"className":6538},[508,545],"1000",[47,6541,6543,6546],{"className":6542},[508,545],[47,6544,2141],{"className":6545},[508,545],[47,6547,6549],{"className":6548},[585],[47,6550,6552],{"className":6551},[520],[47,6553,6555],{"className":6554},[525],[47,6556,6559],{"className":6557,"style":6558},[529],"height:0.8932em;",[47,6560,6562,6566],{"style":6561},"top:-2.8932em;margin-right:0.0714em;",[47,6563],{"className":6564,"style":6565},[537],"height:2.5357em;",[47,6567,6569],{"className":6568},[542,606,607,545],[47,6570,6572,6575,6578,6581],{"className":6571},[508,545],[47,6573,3192],{"className":6574},[508,545],[47,6576,6239],{"className":6577,"style":6259},[508,581,545],[47,6579,141],{"className":6580},[508,545],[47,6582,6584,6587],{"className":6583},[508,545],[47,6585,482],{"className":6586},[508,581,545],[47,6588,6590],{"className":6589},[585],[47,6591,6593,6627],{"className":6592},[520,521],[47,6594,6596,6624],{"className":6595},[525],[47,6597,6599],{"className":6598,"style":595},[529],[47,6600,6602,6606],{"style":6601},"top:-2.3448em;margin-left:0em;margin-right:0.1em;",[47,6603],{"className":6604,"style":6605},[537],"height:2.6944em;",[47,6607,6609,6612,6615,6618,6621],{"className":6608},[508,545],[47,6610,755],{"className":6611},[508,581,545],[47,6613,1181],{"className":6614},[508,581,545],[47,6616,482],{"className":6617},[508,581,545],[47,6619,1171],{"className":6620},[508,581,545],[47,6622,3273],{"className":6623,"style":5726},[508,581,545],[47,6625,616],{"className":6626},[615],[47,6628,6630],{"className":6629},[525],[47,6631,6634],{"className":6632,"style":6633},[529],"height:0.3496em;",[47,6635],{},[47,6637,6638,6641],{"style":663},[47,6639],{"className":6640,"style":538},[537],[47,6642],{"className":6643,"style":671},[670],[47,6645,6646,6649],{"style":674},[47,6647],{"className":6648,"style":538},[537],[47,6650,6652],{"className":6651},[542,543,544,545],[47,6653,6655],{"className":6654},[508,545],[47,6656,1178],{"className":6657},[508,581,545],[47,6659,616],{"className":6660},[615],[47,6662,6664],{"className":6663},[525],[47,6665,6668],{"className":6666,"style":6667},[529],"height:0.4352em;",[47,6669],{},[47,6671],{"className":6672},[702,513],[47,6674,6676],{"className":6675,"style":1354},[702,1353],[47,6677,1201],{"className":6678},[1358,1359],[14,6680,6681],{},[47,6682,6684,6765],{"className":6683},[451],[47,6685,6687],{"className":6686},[455],[457,6688,6689],{"xmlns":459},[461,6690,6691,6762],{},[464,6692,6693,6695,6717,6719,6722,6724],{},[480,6694,5974],{},[477,6696,6697,6699],{},[480,6698,5977],{},[464,6700,6701,6703,6705,6707,6709,6711,6713,6715],{},[850,6702,1187],{"stretchy":1186},[480,6704,1178],{},[850,6706,1192],{"separator":495},[470,6708,3192],{},[480,6710,6239],{},[850,6712,6147],{},[470,6714,472],{},[850,6716,1201],{"stretchy":1186},[850,6718,82],{},[480,6720,6721],{},"cos",[850,6723,1210],{},[464,6725,6726,6728,6760],{},[850,6727,1187],{"fence":495},[467,6729,6730,6732],{},[480,6731,1178],{},[854,6733,6734,6736],{},[470,6735,6371],{},[464,6737,6738,6740,6742,6744],{},[470,6739,3192],{},[480,6741,6239],{},[480,6743,141],{"mathvariant":1206},[477,6745,6746,6748],{},[480,6747,482],{},[464,6749,6750,6752,6754,6756,6758],{},[480,6751,755],{},[480,6753,1181],{},[480,6755,482],{},[480,6757,1171],{},[480,6759,3273],{},[850,6761,1201],{"fence":495},[487,6763,6764],{"encoding":489}," PE_{(i,2j+1)} = \\cos\\left(\\frac{i}{10000^{2j/d_{model}}}\\right)",[47,6766,6768,6850],{"className":6767,"ariaHidden":495},[494],[47,6769,6771,6774,6777,6841,6844,6847],{"className":6770},[499],[47,6772],{"className":6773,"style":6410},[503],[47,6775,5974],{"className":6776,"style":3555},[508,581],[47,6778,6780,6783],{"className":6779},[508],[47,6781,5977],{"className":6782,"style":6420},[508,581],[47,6784,6786],{"className":6785},[585],[47,6787,6789,6833],{"className":6788},[520,521],[47,6790,6792,6830],{"className":6791},[525],[47,6793,6795],{"className":6794,"style":595},[529],[47,6796,6797,6800],{"style":6435},[47,6798],{"className":6799,"style":929},[537],[47,6801,6803],{"className":6802},[542,543,544,545],[47,6804,6806,6809,6812,6815,6818,6821,6824,6827],{"className":6805},[508,545],[47,6807,1187],{"className":6808},[512,545],[47,6810,1178],{"className":6811},[508,581,545],[47,6813,1192],{"className":6814},[1299,545],[47,6816,3192],{"className":6817},[508,545],[47,6819,6239],{"className":6820,"style":6259},[508,581,545],[47,6822,6147],{"className":6823},[942,545],[47,6825,472],{"className":6826},[508,545],[47,6828,1201],{"className":6829},[702,545],[47,6831,616],{"className":6832},[615],[47,6834,6836],{"className":6835},[525],[47,6837,6839],{"className":6838,"style":6472},[529],[47,6840],{},[47,6842],{"className":6843,"style":889},[888],[47,6845,82],{"className":6846},[893],[47,6848],{"className":6849,"style":889},[888],[47,6851,6853,6856,6859,6862],{"className":6852},[499],[47,6854],{"className":6855,"style":1334},[503],[47,6857,6721],{"className":6858},[1338],[47,6860],{"className":6861,"style":1303},[888],[47,6863,6865,6871,7029],{"className":6864},[1349],[47,6866,6868],{"className":6867,"style":1354},[512,1353],[47,6869,1187],{"className":6870},[1358,1359],[47,6872,6874,6877,7026],{"className":6873},[508],[47,6875],{"className":6876},[512,513],[47,6878,6880],{"className":6879},[467],[47,6881,6883,7018],{"className":6882},[520,521],[47,6884,6886,7015],{"className":6885},[525],[47,6887,6889,6993,7001],{"className":6888,"style":6523},[529],[47,6890,6891,6894],{"style":6526},[47,6892],{"className":6893,"style":538},[537],[47,6895,6897],{"className":6896},[542,543,544,545],[47,6898,6900,6903],{"className":6899},[508,545],[47,6901,6539],{"className":6902},[508,545],[47,6904,6906,6909],{"className":6905},[508,545],[47,6907,2141],{"className":6908},[508,545],[47,6910,6912],{"className":6911},[585],[47,6913,6915],{"className":6914},[520],[47,6916,6918],{"className":6917},[525],[47,6919,6921],{"className":6920,"style":6558},[529],[47,6922,6923,6926],{"style":6561},[47,6924],{"className":6925,"style":6565},[537],[47,6927,6929],{"className":6928},[542,606,607,545],[47,6930,6932,6935,6938,6941],{"className":6931},[508,545],[47,6933,3192],{"className":6934},[508,545],[47,6936,6239],{"className":6937,"style":6259},[508,581,545],[47,6939,141],{"className":6940},[508,545],[47,6942,6944,6947],{"className":6943},[508,545],[47,6945,482],{"className":6946},[508,581,545],[47,6948,6950],{"className":6949},[585],[47,6951,6953,6985],{"className":6952},[520,521],[47,6954,6956,6982],{"className":6955},[525],[47,6957,6959],{"className":6958,"style":595},[529],[47,6960,6961,6964],{"style":6601},[47,6962],{"className":6963,"style":6605},[537],[47,6965,6967,6970,6973,6976,6979],{"className":6966},[508,545],[47,6968,755],{"className":6969},[508,581,545],[47,6971,1181],{"className":6972},[508,581,545],[47,6974,482],{"className":6975},[508,581,545],[47,6977,1171],{"className":6978},[508,581,545],[47,6980,3273],{"className":6981,"style":5726},[508,581,545],[47,6983,616],{"className":6984},[615],[47,6986,6988],{"className":6987},[525],[47,6989,6991],{"className":6990,"style":6633},[529],[47,6992],{},[47,6994,6995,6998],{"style":663},[47,6996],{"className":6997,"style":538},[537],[47,6999],{"className":7000,"style":671},[670],[47,7002,7003,7006],{"style":674},[47,7004],{"className":7005,"style":538},[537],[47,7007,7009],{"className":7008},[542,543,544,545],[47,7010,7012],{"className":7011},[508,545],[47,7013,1178],{"className":7014},[508,581,545],[47,7016,616],{"className":7017},[615],[47,7019,7021],{"className":7020},[525],[47,7022,7024],{"className":7023,"style":6667},[529],[47,7025],{},[47,7027],{"className":7028},[702,513],[47,7030,7032],{"className":7031,"style":1354},[702,1353],[47,7033,1201],{"className":7034},[1358,1359],[14,7036,7037,7038,7066,7067,7095,7096,7131,7132,7188,7189,7217,7218,7311,7312,7391],{},"其中，",[47,7039,7041,7054],{"className":7040},[451],[47,7042,7044],{"className":7043},[455],[457,7045,7046],{"xmlns":459},[461,7047,7048,7052],{},[464,7049,7050],{},[480,7051,1178],{},[487,7053,1178],{"encoding":489},[47,7055,7057],{"className":7056,"ariaHidden":495},[494],[47,7058,7060,7063],{"className":7059},[499],[47,7061],{"className":7062,"style":6218},[503],[47,7064,1178],{"className":7065},[508,581]," 表示位置，",[47,7068,7070,7083],{"className":7069},[451],[47,7071,7073],{"className":7072},[455],[457,7074,7075],{"xmlns":459},[461,7076,7077,7081],{},[464,7078,7079],{},[480,7080,6239],{},[487,7082,6239],{"encoding":489},[47,7084,7086],{"className":7085,"ariaHidden":495},[494],[47,7087,7089,7092],{"className":7088},[499],[47,7090],{"className":7091,"style":6252},[503],[47,7093,6239],{"className":7094,"style":6259},[508,581]," 表示维度。也就是说，位置编码的每一个维度都对应一个正弦曲线。其波长从 ",[47,7097,7099,7116],{"className":7098},[451],[47,7100,7102],{"className":7101},[455],[457,7103,7104],{"xmlns":459},[461,7105,7106,7113],{},[464,7107,7108,7110],{},[470,7109,3192],{},[480,7111,7112],{},"π",[487,7114,7115],{"encoding":489},"2\\pi",[47,7117,7119],{"className":7118,"ariaHidden":495},[494],[47,7120,7122,7125,7128],{"className":7121},[499],[47,7123],{"className":7124,"style":2153},[503],[47,7126,3192],{"className":7127},[508],[47,7129,7112],{"className":7130,"style":830},[508,581]," 到 ",[47,7133,7135,7155],{"className":7134},[451],[47,7136,7138],{"className":7137},[455],[457,7139,7140],{"xmlns":459},[461,7141,7142,7152],{},[464,7143,7144,7146,7148,7150],{},[470,7145,6371],{},[850,7147,2203],{},[470,7149,3192],{},[480,7151,7112],{},[487,7153,7154],{"encoding":489},"10000 \\cdot 2\\pi",[47,7156,7158,7176],{"className":7157,"ariaHidden":495},[494],[47,7159,7161,7164,7167,7170,7173],{"className":7160},[499],[47,7162],{"className":7163,"style":2153},[503],[47,7165,6371],{"className":7166},[508],[47,7168],{"className":7169,"style":2259},[888],[47,7171,2203],{"className":7172},[942],[47,7174],{"className":7175,"style":2259},[888],[47,7177,7179,7182,7185],{"className":7178},[499],[47,7180],{"className":7181,"style":2153},[503],[47,7183,3192],{"className":7184},[508],[47,7186,7112],{"className":7187,"style":830},[508,581]," 按几何级数增长。我们选择这个函数，是因为我们假设它能够使模型更容易学习关注相对位置，因为对于任意固定的偏移量 ",[47,7190,7192,7205],{"className":7191},[451],[47,7193,7195],{"className":7194},[455],[457,7196,7197],{"xmlns":459},[461,7198,7199,7203],{},[464,7200,7201],{},[480,7202,485],{},[487,7204,485],{"encoding":489},[47,7206,7208],{"className":7207,"ariaHidden":495},[494],[47,7209,7211,7214],{"className":7210},[499],[47,7212],{"className":7213,"style":796},[503],[47,7215,485],{"className":7216,"style":611},[508,581],"，",[47,7219,7221,7247],{"className":7220},[451],[47,7222,7224],{"className":7223},[455],[457,7225,7226],{"xmlns":459},[461,7227,7228,7244],{},[464,7229,7230,7232],{},[480,7231,5974],{},[477,7233,7234,7236],{},[480,7235,5977],{},[464,7237,7238,7240,7242],{},[480,7239,1178],{},[850,7241,6147],{},[480,7243,485],{},[487,7245,7246],{"encoding":489},"PE_{i+k}",[47,7248,7250],{"className":7249,"ariaHidden":495},[494],[47,7251,7253,7257,7260],{"className":7252},[499],[47,7254],{"className":7255,"style":7256},[503],"height:0.8917em;vertical-align:-0.2083em;",[47,7258,5974],{"className":7259,"style":3555},[508,581],[47,7261,7263,7266],{"className":7262},[508],[47,7264,5977],{"className":7265,"style":6420},[508,581],[47,7267,7269],{"className":7268},[585],[47,7270,7272,7302],{"className":7271},[520,521],[47,7273,7275,7299],{"className":7274},[525],[47,7276,7278],{"className":7277,"style":1635},[529],[47,7279,7281,7284],{"style":7280},"top:-2.55em;margin-left:-0.0576em;margin-right:0.05em;",[47,7282],{"className":7283,"style":929},[537],[47,7285,7287],{"className":7286},[542,543,544,545],[47,7288,7290,7293,7296],{"className":7289},[508,545],[47,7291,1178],{"className":7292},[508,581,545],[47,7294,6147],{"className":7295},[942,545],[47,7297,485],{"className":7298,"style":611},[508,581,545],[47,7300,616],{"className":7301},[615],[47,7303,7305],{"className":7304},[525],[47,7306,7309],{"className":7307,"style":7308},[529],"height:0.2083em;",[47,7310],{}," 都可以表示为 ",[47,7313,7315,7335],{"className":7314},[451],[47,7316,7318],{"className":7317},[455],[457,7319,7320],{"xmlns":459},[461,7321,7322,7332],{},[464,7323,7324,7326],{},[480,7325,5974],{},[477,7327,7328,7330],{},[480,7329,5977],{},[480,7331,1178],{},[487,7333,7334],{"encoding":489},"PE_{i}",[47,7336,7338],{"className":7337,"ariaHidden":495},[494],[47,7339,7341,7345,7348],{"className":7340},[499],[47,7342],{"className":7343,"style":7344},[503],"height:0.8333em;vertical-align:-0.15em;",[47,7346,5974],{"className":7347,"style":3555},[508,581],[47,7349,7351,7354],{"className":7350},[508],[47,7352,5977],{"className":7353,"style":6420},[508,581],[47,7355,7357],{"className":7356},[585],[47,7358,7360,7383],{"className":7359},[520,521],[47,7361,7363,7380],{"className":7362},[525],[47,7364,7366],{"className":7365,"style":2419},[529],[47,7367,7368,7371],{"style":7280},[47,7369],{"className":7370,"style":929},[537],[47,7372,7374],{"className":7373},[542,543,544,545],[47,7375,7377],{"className":7376},[508,545],[47,7378,1178],{"className":7379},[508,581,545],[47,7381,616],{"className":7382},[615],[47,7384,7386],{"className":7385},[525],[47,7387,7389],{"className":7388,"style":1657},[529],[47,7390],{}," 的线性函数。",[37,7393,7395],{"className":2778,"code":7394,"language":2780,"meta":42,"style":42},"class PositionalEncoding(nn.Block):\n    \"\"\"位置编码\"\"\"\n    def __init__(self, num_hiddens, dropout, max_len=1000):\n        super(PositionalEncoding, self).__init__()\n        self.dropout = nn.Dropout(dropout)\n        # 创建一个足够长的P\n        self.P = np.zeros((1, max_len, num_hiddens))\n        X = np.arange(max_len).reshape(-1, 1) / np.power(\n            10000, np.arange(0, num_hiddens, 2) / num_hiddens)\n        self.P[:, :, 0::2] = np.sin(X)\n        self.P[:, :, 1::2] = np.cos(X)\n\n    def forward(self, X):\n        X = X + self.P[:, :X.shape[1], :].as_in_ctx(X.ctx)\n        return self.dropout(X)\n",[44,7396,7397,7415,7420,7449,7465,7475,7480,7497,7522,7544,7576,7603,7607,7623,7656],{"__ignoreMap":42},[47,7398,7399,7401,7404,7406,7408,7410,7413],{"class":49,"line":50},[47,7400,3003],{"class":81},[47,7402,7403],{"class":3006}," PositionalEncoding",[47,7405,1187],{"class":353},[47,7407,3013],{"class":3012},[47,7409,144],{"class":353},[47,7411,7412],{"class":3012},"Block",[47,7414,2805],{"class":353},[47,7416,7417],{"class":49,"line":64},[47,7418,7419],{"class":2810},"    \"\"\"位置编码\"\"\"\n",[47,7421,7422,7424,7426,7428,7430,7432,7434,7436,7438,7440,7443,7445,7447],{"class":49,"line":74},[47,7423,3030],{"class":81},[47,7425,3033],{"class":118},[47,7427,1187],{"class":353},[47,7429,3039],{"class":3038},[47,7431,2799],{"class":353},[47,7433,5232],{"class":2795},[47,7435,2799],{"class":353},[47,7437,3044],{"class":2795},[47,7439,2799],{"class":353},[47,7441,7442],{"class":2795},"max_len",[47,7444,82],{"class":81},[47,7446,6539],{"class":118},[47,7448,2805],{"class":353},[47,7450,7451,7453,7456,7458,7460,7462],{"class":49,"line":335},[47,7452,3060],{"class":3059},[47,7454,7455],{"class":353},"(PositionalEncoding, ",[47,7457,3039],{"class":3066},[47,7459,3069],{"class":353},[47,7461,3072],{"class":118},[47,7463,7464],{"class":353},"()\n",[47,7466,7467,7469,7471,7473],{"class":49,"line":344},[47,7468,3084],{"class":3066},[47,7470,3087],{"class":353},[47,7472,82],{"class":81},[47,7474,3092],{"class":353},[47,7476,7477],{"class":49,"line":2856},[47,7478,7479],{"class":331},"        # 创建一个足够长的P\n",[47,7481,7482,7484,7487,7489,7492,7494],{"class":49,"line":2864},[47,7483,3084],{"class":3066},[47,7485,7486],{"class":353},".P ",[47,7488,82],{"class":81},[47,7490,7491],{"class":353}," np.zeros((",[47,7493,472],{"class":118},[47,7495,7496],{"class":353},", max_len, num_hiddens))\n",[47,7498,7499,7501,7503,7506,7508,7510,7512,7514,7517,7519],{"class":49,"line":2875},[47,7500,2942],{"class":353},[47,7502,82],{"class":81},[47,7504,7505],{"class":353}," np.arange(max_len).reshape(",[47,7507,2926],{"class":81},[47,7509,472],{"class":118},[47,7511,2799],{"class":353},[47,7513,472],{"class":118},[47,7515,7516],{"class":353},") ",[47,7518,141],{"class":81},[47,7520,7521],{"class":353}," np.power(\n",[47,7523,7524,7527,7530,7532,7535,7537,7539,7541],{"class":49,"line":2892},[47,7525,7526],{"class":118},"            10000",[47,7528,7529],{"class":353},", np.arange(",[47,7531,2141],{"class":118},[47,7533,7534],{"class":353},", num_hiddens, ",[47,7536,3192],{"class":118},[47,7538,7516],{"class":353},[47,7540,141],{"class":81},[47,7542,7543],{"class":353}," num_hiddens)\n",[47,7545,7546,7548,7551,7555,7557,7559,7561,7563,7566,7568,7571,7573],{"class":49,"line":2908},[47,7547,3084],{"class":3066},[47,7549,7550],{"class":353},".P[",[47,7552,7554],{"class":7553},"sDoOe",":",[47,7556,2799],{"class":353},[47,7558,7554],{"class":7553},[47,7560,2799],{"class":353},[47,7562,2141],{"class":118},[47,7564,7565],{"class":7553},"::",[47,7567,3192],{"class":118},[47,7569,7570],{"class":353},"] ",[47,7572,82],{"class":81},[47,7574,7575],{"class":353}," np.sin(X)\n",[47,7577,7578,7580,7582,7584,7586,7588,7590,7592,7594,7596,7598,7600],{"class":49,"line":2916},[47,7579,3084],{"class":3066},[47,7581,7550],{"class":353},[47,7583,7554],{"class":7553},[47,7585,2799],{"class":353},[47,7587,7554],{"class":7553},[47,7589,2799],{"class":353},[47,7591,472],{"class":118},[47,7593,7565],{"class":7553},[47,7595,3192],{"class":118},[47,7597,7570],{"class":353},[47,7599,82],{"class":81},[47,7601,7602],{"class":353}," np.cos(X)\n",[47,7604,7605],{"class":49,"line":2933},[47,7606,326],{"emptyLinePlaceholder":325},[47,7608,7609,7611,7613,7615,7617,7619,7621],{"class":49,"line":2939},[47,7610,3030],{"class":81},[47,7612,3123],{"class":53},[47,7614,1187],{"class":353},[47,7616,3039],{"class":3038},[47,7618,2799],{"class":353},[47,7620,2796],{"class":2795},[47,7622,2805],{"class":353},[47,7624,7625,7627,7629,7632,7634,7636,7638,7640,7642,7644,7647,7649,7651,7653],{"class":49,"line":2964},[47,7626,2942],{"class":353},[47,7628,82],{"class":81},[47,7630,7631],{"class":353}," X ",[47,7633,6147],{"class":81},[47,7635,5590],{"class":3066},[47,7637,7550],{"class":353},[47,7639,7554],{"class":7553},[47,7641,2799],{"class":353},[47,7643,7554],{"class":7553},[47,7645,7646],{"class":353},"X.shape[",[47,7648,472],{"class":118},[47,7650,5164],{"class":353},[47,7652,7554],{"class":7553},[47,7654,7655],{"class":353},"].as_in_ctx(X.ctx)\n",[47,7657,7658,7660,7662],{"class":49,"line":2977},[47,7659,2838],{"class":81},[47,7661,5590],{"class":3066},[47,7663,7664],{"class":353},".dropout(X)\n",[10,7666,7667],{"id":7667},"前馈神经网络",[14,7669,7670],{},"除了注意力子层外，我们的编码器和解码器的每一层中都包含一个全连接的前馈神经网络，该网络分别且相同地应用于每个位置。这包括两个线性变换，中间使用ReLU激活函数。",[14,7672,7673],{},[47,7674,7676,7749],{"className":7675},[451],[47,7677,7679],{"className":7678},[455],[457,7680,7681],{"xmlns":459},[461,7682,7683,7746],{},[464,7684,7685,7688,7690,7693,7695,7698,7700,7702,7705,7707,7709,7711,7713,7715,7721,7723,7730,7732,7738,7740],{},[480,7686,7687],{},"F",[480,7689,7687],{},[480,7691,7692],{},"N",[850,7694,1187],{"stretchy":1186},[480,7696,7697],{},"x",[850,7699,1201],{"stretchy":1186},[850,7701,82],{},[480,7703,7704],{},"max",[850,7706,1210],{},[850,7708,1187],{"stretchy":1186},[470,7710,2141],{},[850,7712,1192],{"separator":495},[480,7714,7697],{},[477,7716,7717,7719],{},[480,7718,3363],{},[470,7720,472],{},[850,7722,6147],{},[477,7724,7725,7728],{},[480,7726,7727],{},"b",[470,7729,472],{},[850,7731,1201],{"stretchy":1186},[477,7733,7734,7736],{},[480,7735,3363],{},[470,7737,3192],{},[850,7739,6147],{},[477,7741,7742,7744],{},[480,7743,7727],{},[470,7745,3192],{},[487,7747,7748],{"encoding":489}," FFN(x) = \\max(0,xW_1+b_1)W_2+b_2",[47,7750,7752,7786,7860,7958],{"className":7751,"ariaHidden":495},[494],[47,7753,7755,7758,7761,7764,7768,7771,7774,7777,7780,7783],{"className":7754},[499],[47,7756],{"className":7757,"style":1264},[503],[47,7759,7687],{"className":7760,"style":3555},[508,581],[47,7762,7687],{"className":7763,"style":3555},[508,581],[47,7765,7692],{"className":7766,"style":7767},[508,581],"margin-right:0.10903em;",[47,7769,1187],{"className":7770},[512],[47,7772,7697],{"className":7773},[508,581],[47,7775,1201],{"className":7776},[702],[47,7778],{"className":7779,"style":889},[888],[47,7781,82],{"className":7782},[893],[47,7784],{"className":7785,"style":889},[888],[47,7787,7789,7792,7795,7798,7801,7804,7807,7810,7851,7854,7857],{"className":7788},[499],[47,7790],{"className":7791,"style":1264},[503],[47,7793,7704],{"className":7794},[1338],[47,7796,1187],{"className":7797},[512],[47,7799,2141],{"className":7800},[508],[47,7802,1192],{"className":7803},[1299],[47,7805],{"className":7806,"style":1303},[888],[47,7808,7697],{"className":7809},[508,581],[47,7811,7813,7816],{"className":7812},[508],[47,7814,3363],{"className":7815,"style":3555},[508,581],[47,7817,7819],{"className":7818},[585],[47,7820,7822,7843],{"className":7821},[520,521],[47,7823,7825,7840],{"className":7824},[525],[47,7826,7828],{"className":7827,"style":3461},[529],[47,7829,7831,7834],{"style":7830},"top:-2.55em;margin-left:-0.1389em;margin-right:0.05em;",[47,7832],{"className":7833,"style":929},[537],[47,7835,7837],{"className":7836},[542,543,544,545],[47,7838,472],{"className":7839},[508,545],[47,7841,616],{"className":7842},[615],[47,7844,7846],{"className":7845},[525],[47,7847,7849],{"className":7848,"style":1657},[529],[47,7850],{},[47,7852],{"className":7853,"style":2259},[888],[47,7855,6147],{"className":7856},[942],[47,7858],{"className":7859,"style":2259},[888],[47,7861,7863,7866,7906,7909,7949,7952,7955],{"className":7862},[499],[47,7864],{"className":7865,"style":1264},[503],[47,7867,7869,7872],{"className":7868},[508],[47,7870,7727],{"className":7871},[508,581],[47,7873,7875],{"className":7874},[585],[47,7876,7878,7898],{"className":7877},[520,521],[47,7879,7881,7895],{"className":7880},[525],[47,7882,7884],{"className":7883,"style":3461},[529],[47,7885,7886,7889],{"style":1638},[47,7887],{"className":7888,"style":929},[537],[47,7890,7892],{"className":7891},[542,543,544,545],[47,7893,472],{"className":7894},[508,545],[47,7896,616],{"className":7897},[615],[47,7899,7901],{"className":7900},[525],[47,7902,7904],{"className":7903,"style":1657},[529],[47,7905],{},[47,7907,1201],{"className":7908},[702],[47,7910,7912,7915],{"className":7911},[508],[47,7913,3363],{"className":7914,"style":3555},[508,581],[47,7916,7918],{"className":7917},[585],[47,7919,7921,7941],{"className":7920},[520,521],[47,7922,7924,7938],{"className":7923},[525],[47,7925,7927],{"className":7926,"style":3461},[529],[47,7928,7929,7932],{"style":7830},[47,7930],{"className":7931,"style":929},[537],[47,7933,7935],{"className":7934},[542,543,544,545],[47,7936,3192],{"className":7937},[508,545],[47,7939,616],{"className":7940},[615],[47,7942,7944],{"className":7943},[525],[47,7945,7947],{"className":7946,"style":1657},[529],[47,7948],{},[47,7950],{"className":7951,"style":2259},[888],[47,7953,6147],{"className":7954},[942],[47,7956],{"className":7957,"style":2259},[888],[47,7959,7961,7964],{"className":7960},[499],[47,7962],{"className":7963,"style":1616},[503],[47,7965,7967,7970],{"className":7966},[508],[47,7968,7727],{"className":7969},[508,581],[47,7971,7973],{"className":7972},[585],[47,7974,7976,7996],{"className":7975},[520,521],[47,7977,7979,7993],{"className":7978},[525],[47,7980,7982],{"className":7981,"style":3461},[529],[47,7983,7984,7987],{"style":1638},[47,7985],{"className":7986,"style":929},[537],[47,7988,7990],{"className":7989},[542,543,544,545],[47,7991,3192],{"className":7992},[508,545],[47,7994,616],{"className":7995},[615],[47,7997,7999],{"className":7998},[525],[47,8000,8002],{"className":8001,"style":1657},[529],[47,8003],{},[37,8005,8007],{"className":2778,"code":8006,"language":2780,"meta":42,"style":42},"class PositionWiseFFN(nn.Module):\n    \"\"\"基于位置的前馈网络\"\"\"\n    def __init__(self, ffn_num_input, ffn_num_hiddens, ffn_num_outputs,\n                 **kwargs):\n        super(PositionWiseFFN, self).__init__(**kwargs)\n        self.dense1 = nn.Linear(ffn_num_input, ffn_num_hiddens)\n        self.relu = nn.ReLU()\n        self.dense2 = nn.Linear(ffn_num_hiddens, ffn_num_outputs)\n\n    def forward(self, X):\n        return self.dense2(self.relu(self.dense1(X)))\n",[44,8008,8009,8026,8031,8058,8067,8086,8098,8110,8122,8126,8142],{"__ignoreMap":42},[47,8010,8011,8013,8016,8018,8020,8022,8024],{"class":49,"line":50},[47,8012,3003],{"class":81},[47,8014,8015],{"class":3006}," PositionWiseFFN",[47,8017,1187],{"class":353},[47,8019,3013],{"class":3012},[47,8021,144],{"class":353},[47,8023,3018],{"class":3012},[47,8025,2805],{"class":353},[47,8027,8028],{"class":49,"line":64},[47,8029,8030],{"class":2810},"    \"\"\"基于位置的前馈网络\"\"\"\n",[47,8032,8033,8035,8037,8039,8041,8043,8046,8048,8051,8053,8056],{"class":49,"line":74},[47,8034,3030],{"class":81},[47,8036,3033],{"class":118},[47,8038,1187],{"class":353},[47,8040,3039],{"class":3038},[47,8042,2799],{"class":353},[47,8044,8045],{"class":2795},"ffn_num_input",[47,8047,2799],{"class":353},[47,8049,8050],{"class":2795},"ffn_num_hiddens",[47,8052,2799],{"class":353},[47,8054,8055],{"class":2795},"ffn_num_outputs",[47,8057,4196],{"class":353},[47,8059,8060,8063,8065],{"class":49,"line":335},[47,8061,8062],{"class":81},"                 **",[47,8064,3052],{"class":2795},[47,8066,2805],{"class":353},[47,8068,8069,8071,8074,8076,8078,8080,8082,8084],{"class":49,"line":344},[47,8070,3060],{"class":3059},[47,8072,8073],{"class":353},"(PositionWiseFFN, ",[47,8075,3039],{"class":3066},[47,8077,3069],{"class":353},[47,8079,3072],{"class":118},[47,8081,1187],{"class":353},[47,8083,3049],{"class":81},[47,8085,3079],{"class":353},[47,8087,8088,8090,8093,8095],{"class":49,"line":2856},[47,8089,3084],{"class":3066},[47,8091,8092],{"class":353},".dense1 ",[47,8094,82],{"class":81},[47,8096,8097],{"class":353}," nn.Linear(ffn_num_input, ffn_num_hiddens)\n",[47,8099,8100,8102,8105,8107],{"class":49,"line":2864},[47,8101,3084],{"class":3066},[47,8103,8104],{"class":353},".relu ",[47,8106,82],{"class":81},[47,8108,8109],{"class":353}," nn.ReLU()\n",[47,8111,8112,8114,8117,8119],{"class":49,"line":2875},[47,8113,3084],{"class":3066},[47,8115,8116],{"class":353},".dense2 ",[47,8118,82],{"class":81},[47,8120,8121],{"class":353}," nn.Linear(ffn_num_hiddens, ffn_num_outputs)\n",[47,8123,8124],{"class":49,"line":2892},[47,8125,326],{"emptyLinePlaceholder":325},[47,8127,8128,8130,8132,8134,8136,8138,8140],{"class":49,"line":2908},[47,8129,3030],{"class":81},[47,8131,3123],{"class":53},[47,8133,1187],{"class":353},[47,8135,3039],{"class":3038},[47,8137,2799],{"class":353},[47,8139,2796],{"class":2795},[47,8141,2805],{"class":353},[47,8143,8144,8146,8148,8151,8153,8156,8158],{"class":49,"line":2916},[47,8145,2838],{"class":81},[47,8147,5590],{"class":3066},[47,8149,8150],{"class":353},".dense2(",[47,8152,3039],{"class":3066},[47,8154,8155],{"class":353},".relu(",[47,8157,3039],{"class":3066},[47,8159,8160],{"class":353},".dense1(X)))\n",[10,8162,8163],{"id":8163},"残差连接和层规范化",[14,8165,8166],{},"我们在每个子层之间加入了一个残差连接和层归一化。",[14,8168,8169,8170,4938,8295,8358],{},"因此，每个子层的输出都是 ",[47,8171,8173,8241],{"className":8172},[451],[47,8174,8176],{"className":8175},[455],[457,8177,8178],{"xmlns":459},[461,8179,8180,8238],{},[464,8181,8182,8205,8207,8209,8211,8230,8232,8234,8236],{},[464,8183,8184,8187,8189,8192,8194,8197,8199,8201,8203],{},[480,8185,8186],{"mathvariant":1206},"L",[480,8188,285],{"mathvariant":1206},[480,8190,8191],{"mathvariant":1206},"y",[480,8193,1171],{"mathvariant":1206},[480,8195,8196],{"mathvariant":1206},"r",[480,8198,7692],{"mathvariant":1206},[480,8200,1181],{"mathvariant":1206},[480,8202,8196],{"mathvariant":1206},[480,8204,755],{"mathvariant":1206},[850,8206,1187],{"stretchy":1186},[480,8208,7697],{},[850,8210,6147],{},[464,8212,8213,8216,8218,8220,8222,8224,8226,8228],{},[480,8214,8215],{"mathvariant":1206},"S",[480,8217,3270],{"mathvariant":1206},[480,8219,7727],{"mathvariant":1206},[480,8221,3273],{"mathvariant":1206},[480,8223,285],{"mathvariant":1206},[480,8225,8191],{"mathvariant":1206},[480,8227,1171],{"mathvariant":1206},[480,8229,8196],{"mathvariant":1206},[850,8231,1187],{"stretchy":1186},[480,8233,7697],{},[850,8235,1201],{"stretchy":1186},[850,8237,1201],{"stretchy":1186},[487,8239,8240],{"encoding":489},"\\mathrm{LayerNorm}(x + \\mathrm{Sublayer}(x))",[47,8242,8244,8272],{"className":8243,"ariaHidden":495},[494],[47,8245,8247,8250,8257,8260,8263,8266,8269],{"className":8246},[499],[47,8248],{"className":8249,"style":1264},[503],[47,8251,8253],{"className":8252},[508],[47,8254,8256],{"className":8255},[508,1342],"LayerNorm",[47,8258,1187],{"className":8259},[512],[47,8261,7697],{"className":8262},[508,581],[47,8264],{"className":8265,"style":2259},[888],[47,8267,6147],{"className":8268},[942],[47,8270],{"className":8271,"style":2259},[888],[47,8273,8275,8278,8285,8288,8291],{"className":8274},[499],[47,8276],{"className":8277,"style":1264},[503],[47,8279,8281],{"className":8280},[508],[47,8282,8284],{"className":8283},[508,1342],"Sublayer",[47,8286,1187],{"className":8287},[512],[47,8289,7697],{"className":8290},[508,581],[47,8292,8294],{"className":8293},[702],"))",[47,8296,8298,8334],{"className":8297},[451],[47,8299,8301],{"className":8300},[455],[457,8302,8303],{"xmlns":459},[461,8304,8305,8331],{},[464,8306,8307,8325,8327,8329],{},[464,8308,8309,8311,8313,8315,8317,8319,8321,8323],{},[480,8310,8215],{"mathvariant":1206},[480,8312,3270],{"mathvariant":1206},[480,8314,7727],{"mathvariant":1206},[480,8316,3273],{"mathvariant":1206},[480,8318,285],{"mathvariant":1206},[480,8320,8191],{"mathvariant":1206},[480,8322,1171],{"mathvariant":1206},[480,8324,8196],{"mathvariant":1206},[850,8326,1187],{"stretchy":1186},[480,8328,7697],{},[850,8330,1201],{"stretchy":1186},[487,8332,8333],{"encoding":489},"\\mathrm{Sublayer}(x)",[47,8335,8337],{"className":8336,"ariaHidden":495},[494],[47,8338,8340,8343,8349,8352,8355],{"className":8339},[499],[47,8341],{"className":8342,"style":1264},[503],[47,8344,8346],{"className":8345},[508],[47,8347,8284],{"className":8348},[508,1342],[47,8350,1187],{"className":8351},[512],[47,8353,7697],{"className":8354},[508,581],[47,8356,1201],{"className":8357},[702]," 是由该子层本身实现的函数。我们在每个子层的输出上应用 dropout，然后再将其加到子层输入并进行归一化。",[37,8360,8362],{"className":2778,"code":8361,"language":2780,"meta":42,"style":42},"class AddNorm(nn.Module):\n    \"\"\"残差连接后进行层规范化\"\"\"\n    def __init__(self, normalized_shape, dropout, **kwargs):\n        super(AddNorm, self).__init__(**kwargs)\n        self.dropout = nn.Dropout(dropout)\n        self.ln = nn.LayerNorm(normalized_shape)\n\n    def forward(self, X, Y):\n        return self.ln(self.dropout(Y) + X)\n",[44,8363,8364,8381,8386,8413,8432,8442,8454,8458,8479],{"__ignoreMap":42},[47,8365,8366,8368,8371,8373,8375,8377,8379],{"class":49,"line":50},[47,8367,3003],{"class":81},[47,8369,8370],{"class":3006}," AddNorm",[47,8372,1187],{"class":353},[47,8374,3013],{"class":3012},[47,8376,144],{"class":353},[47,8378,3018],{"class":3012},[47,8380,2805],{"class":353},[47,8382,8383],{"class":49,"line":64},[47,8384,8385],{"class":2810},"    \"\"\"残差连接后进行层规范化\"\"\"\n",[47,8387,8388,8390,8392,8394,8396,8398,8401,8403,8405,8407,8409,8411],{"class":49,"line":74},[47,8389,3030],{"class":81},[47,8391,3033],{"class":118},[47,8393,1187],{"class":353},[47,8395,3039],{"class":3038},[47,8397,2799],{"class":353},[47,8399,8400],{"class":2795},"normalized_shape",[47,8402,2799],{"class":353},[47,8404,3044],{"class":2795},[47,8406,2799],{"class":353},[47,8408,3049],{"class":81},[47,8410,3052],{"class":2795},[47,8412,2805],{"class":353},[47,8414,8415,8417,8420,8422,8424,8426,8428,8430],{"class":49,"line":335},[47,8416,3060],{"class":3059},[47,8418,8419],{"class":353},"(AddNorm, ",[47,8421,3039],{"class":3066},[47,8423,3069],{"class":353},[47,8425,3072],{"class":118},[47,8427,1187],{"class":353},[47,8429,3049],{"class":81},[47,8431,3079],{"class":353},[47,8433,8434,8436,8438,8440],{"class":49,"line":344},[47,8435,3084],{"class":3066},[47,8437,3087],{"class":353},[47,8439,82],{"class":81},[47,8441,3092],{"class":353},[47,8443,8444,8446,8449,8451],{"class":49,"line":2856},[47,8445,3084],{"class":3066},[47,8447,8448],{"class":353},".ln ",[47,8450,82],{"class":81},[47,8452,8453],{"class":353}," nn.LayerNorm(normalized_shape)\n",[47,8455,8456],{"class":49,"line":2864},[47,8457,326],{"emptyLinePlaceholder":325},[47,8459,8460,8462,8464,8466,8468,8470,8472,8474,8477],{"class":49,"line":2875},[47,8461,3030],{"class":81},[47,8463,3123],{"class":53},[47,8465,1187],{"class":353},[47,8467,3039],{"class":3038},[47,8469,2799],{"class":353},[47,8471,2796],{"class":2795},[47,8473,2799],{"class":353},[47,8475,8476],{"class":2795},"Y",[47,8478,2805],{"class":353},[47,8480,8481,8483,8485,8488,8490,8493,8495],{"class":49,"line":2892},[47,8482,2838],{"class":81},[47,8484,5590],{"class":3066},[47,8486,8487],{"class":353},".ln(",[47,8489,3039],{"class":3066},[47,8491,8492],{"class":353},".dropout(Y) ",[47,8494,6147],{"class":81},[47,8496,8497],{"class":353}," X)\n",[10,8499,8500],{"id":8500},"编码器-解码器架构",[14,8502,8503,8504,8654,8655,8831,8832,8861,8862,9012],{},"大多数主流的神经序列转换模型都采用编码器—解码器结构。其中，编码器将符号表示组成的输入序列 ",[47,8505,8507,8541],{"className":8506},[451],[47,8508,8510],{"className":8509},[455],[457,8511,8512],{"xmlns":459},[461,8513,8514,8538],{},[464,8515,8516,8518,8524,8526,8528,8530,8536],{},[850,8517,1187],{"stretchy":1186},[477,8519,8520,8522],{},[480,8521,7697],{},[470,8523,472],{},[850,8525,1192],{"separator":495},[850,8527,3340],{},[850,8529,1192],{"separator":495},[477,8531,8532,8534],{},[480,8533,7697],{},[480,8535,724],{},[850,8537,1201],{"stretchy":1186},[487,8539,8540],{"encoding":489},"(x_1, \\ldots, x_n)",[47,8542,8544],{"className":8543,"ariaHidden":495},[494],[47,8545,8547,8550,8553,8593,8596,8599,8602,8605,8608,8611,8651],{"className":8546},[499],[47,8548],{"className":8549,"style":1264},[503],[47,8551,1187],{"className":8552},[512],[47,8554,8556,8559],{"className":8555},[508],[47,8557,7697],{"className":8558},[508,581],[47,8560,8562],{"className":8561},[585],[47,8563,8565,8585],{"className":8564},[520,521],[47,8566,8568,8582],{"className":8567},[525],[47,8569,8571],{"className":8570,"style":3461},[529],[47,8572,8573,8576],{"style":1638},[47,8574],{"className":8575,"style":929},[537],[47,8577,8579],{"className":8578},[542,543,544,545],[47,8580,472],{"className":8581},[508,545],[47,8583,616],{"className":8584},[615],[47,8586,8588],{"className":8587},[525],[47,8589,8591],{"className":8590,"style":1657},[529],[47,8592],{},[47,8594,1192],{"className":8595},[1299],[47,8597],{"className":8598,"style":1303},[888],[47,8600,3340],{"className":8601},[1349],[47,8603],{"className":8604,"style":1303},[888],[47,8606,1192],{"className":8607},[1299],[47,8609],{"className":8610,"style":1303},[888],[47,8612,8614,8617],{"className":8613},[508],[47,8615,7697],{"className":8616},[508,581],[47,8618,8620],{"className":8619},[585],[47,8621,8623,8643],{"className":8622},[520,521],[47,8624,8626,8640],{"className":8625},[525],[47,8627,8629],{"className":8628,"style":1709},[529],[47,8630,8631,8634],{"style":1638},[47,8632],{"className":8633,"style":929},[537],[47,8635,8637],{"className":8636},[542,543,544,545],[47,8638,724],{"className":8639},[508,581,545],[47,8641,616],{"className":8642},[615],[47,8644,8646],{"className":8645},[525],[47,8647,8649],{"className":8648,"style":1657},[529],[47,8650],{},[47,8652,1201],{"className":8653},[702]," 映射为连续表示序列 ",[47,8656,8658,8697],{"className":8657},[451],[47,8659,8661],{"className":8660},[455],[457,8662,8663],{"xmlns":459},[461,8664,8665,8694],{},[464,8666,8667,8670,8672,8674,8680,8682,8684,8686,8692],{},[480,8668,8669],{"mathvariant":5748},"z",[850,8671,82],{},[850,8673,1187],{"stretchy":1186},[477,8675,8676,8678],{},[480,8677,8669],{},[470,8679,472],{},[850,8681,1192],{"separator":495},[850,8683,3340],{},[850,8685,1192],{"separator":495},[477,8687,8688,8690],{},[480,8689,8669],{},[480,8691,724],{},[850,8693,1201],{"stretchy":1186},[487,8695,8696],{"encoding":489},"\\mathbf{z} = (z_1, \\ldots, z_n)",[47,8698,8700,8719],{"className":8699,"ariaHidden":495},[494],[47,8701,8703,8707,8710,8713,8716],{"className":8702},[499],[47,8704],{"className":8705,"style":8706},[503],"height:0.4444em;",[47,8708,8669],{"className":8709},[508,5793],[47,8711],{"className":8712,"style":889},[888],[47,8714,82],{"className":8715},[893],[47,8717],{"className":8718,"style":889},[888],[47,8720,8722,8725,8728,8770,8773,8776,8779,8782,8785,8788,8828],{"className":8721},[499],[47,8723],{"className":8724,"style":1264},[503],[47,8726,1187],{"className":8727},[512],[47,8729,8731,8735],{"className":8730},[508],[47,8732,8669],{"className":8733,"style":8734},[508,581],"margin-right:0.04398em;",[47,8736,8738],{"className":8737},[585],[47,8739,8741,8762],{"className":8740},[520,521],[47,8742,8744,8759],{"className":8743},[525],[47,8745,8747],{"className":8746,"style":3461},[529],[47,8748,8750,8753],{"style":8749},"top:-2.55em;margin-left:-0.044em;margin-right:0.05em;",[47,8751],{"className":8752,"style":929},[537],[47,8754,8756],{"className":8755},[542,543,544,545],[47,8757,472],{"className":8758},[508,545],[47,8760,616],{"className":8761},[615],[47,8763,8765],{"className":8764},[525],[47,8766,8768],{"className":8767,"style":1657},[529],[47,8769],{},[47,8771,1192],{"className":8772},[1299],[47,8774],{"className":8775,"style":1303},[888],[47,8777,3340],{"className":8778},[1349],[47,8780],{"className":8781,"style":1303},[888],[47,8783,1192],{"className":8784},[1299],[47,8786],{"className":8787,"style":1303},[888],[47,8789,8791,8794],{"className":8790},[508],[47,8792,8669],{"className":8793,"style":8734},[508,581],[47,8795,8797],{"className":8796},[585],[47,8798,8800,8820],{"className":8799},[520,521],[47,8801,8803,8817],{"className":8802},[525],[47,8804,8806],{"className":8805,"style":1709},[529],[47,8807,8808,8811],{"style":8749},[47,8809],{"className":8810,"style":929},[537],[47,8812,8814],{"className":8813},[542,543,544,545],[47,8815,724],{"className":8816},[508,581,545],[47,8818,616],{"className":8819},[615],[47,8821,8823],{"className":8822},[525],[47,8824,8826],{"className":8825,"style":1657},[529],[47,8827],{},[47,8829,1201],{"className":8830},[702],"。给定 ",[47,8833,8835,8849],{"className":8834},[451],[47,8836,8838],{"className":8837},[455],[457,8839,8840],{"xmlns":459},[461,8841,8842,8846],{},[464,8843,8844],{},[480,8845,8669],{"mathvariant":5748},[487,8847,8848],{"encoding":489},"\\mathbf{z}",[47,8850,8852],{"className":8851,"ariaHidden":495},[494],[47,8853,8855,8858],{"className":8854},[499],[47,8856],{"className":8857,"style":8706},[503],[47,8859,8669],{"className":8860},[508,5793],"，解码器随后一次生成一个元素的输出符号序列 ",[47,8863,8865,8899],{"className":8864},[451],[47,8866,8868],{"className":8867},[455],[457,8869,8870],{"xmlns":459},[461,8871,8872,8896],{},[464,8873,8874,8876,8882,8884,8886,8888,8894],{},[850,8875,1187],{"stretchy":1186},[477,8877,8878,8880],{},[480,8879,8191],{},[470,8881,472],{},[850,8883,1192],{"separator":495},[850,8885,3340],{},[850,8887,1192],{"separator":495},[477,8889,8890,8892],{},[480,8891,8191],{},[480,8893,755],{},[850,8895,1201],{"stretchy":1186},[487,8897,8898],{"encoding":489},"(y_1, \\ldots, y_m)",[47,8900,8902],{"className":8901,"ariaHidden":495},[494],[47,8903,8905,8908,8911,8951,8954,8957,8960,8963,8966,8969,9009],{"className":8904},[499],[47,8906],{"className":8907,"style":1264},[503],[47,8909,1187],{"className":8910},[512],[47,8912,8914,8917],{"className":8913},[508],[47,8915,8191],{"className":8916,"style":830},[508,581],[47,8918,8920],{"className":8919},[585],[47,8921,8923,8943],{"className":8922},[520,521],[47,8924,8926,8940],{"className":8925},[525],[47,8927,8929],{"className":8928,"style":3461},[529],[47,8930,8931,8934],{"style":2422},[47,8932],{"className":8933,"style":929},[537],[47,8935,8937],{"className":8936},[542,543,544,545],[47,8938,472],{"className":8939},[508,545],[47,8941,616],{"className":8942},[615],[47,8944,8946],{"className":8945},[525],[47,8947,8949],{"className":8948,"style":1657},[529],[47,8950],{},[47,8952,1192],{"className":8953},[1299],[47,8955],{"className":8956,"style":1303},[888],[47,8958,3340],{"className":8959},[1349],[47,8961],{"className":8962,"style":1303},[888],[47,8964,1192],{"className":8965},[1299],[47,8967],{"className":8968,"style":1303},[888],[47,8970,8972,8975],{"className":8971},[508],[47,8973,8191],{"className":8974,"style":830},[508,581],[47,8976,8978],{"className":8977},[585],[47,8979,8981,9001],{"className":8980},[520,521],[47,8982,8984,8998],{"className":8983},[525],[47,8985,8987],{"className":8986,"style":1709},[529],[47,8988,8989,8992],{"style":2422},[47,8990],{"className":8991,"style":929},[537],[47,8993,8995],{"className":8994},[542,543,544,545],[47,8996,755],{"className":8997},[508,581,545],[47,8999,616],{"className":9000},[615],[47,9002,9004],{"className":9003},[525],[47,9005,9007],{"className":9006,"style":1657},[529],[47,9008],{},[47,9010,1201],{"className":9011},[702],"。在每一个时间步，模型都是自回归的，即在生成下一个符号时，会将之前已经生成的符号作为额外输入。",[37,9014,9016],{"className":2778,"code":9015,"language":2780,"meta":42,"style":42},"class Encoder(nn.Module):\n    \"\"\"编码器-解码器架构的基本编码器接口\"\"\"\n    def __init__(self, **kwargs):\n        super(Encoder, self).__init__(**kwargs)\n\n    def forward(self, X, *args):\n        raise NotImplementedError\n\nclass Decoder(nn.Module):\n    \"\"\"编码器-解码器架构的基本解码器接口\"\"\"\n    def __init__(self, **kwargs):\n        super(Decoder, self).__init__(**kwargs)\n\n    def init_state(self, enc_outputs, *args):\n        raise NotImplementedError\n\n    def forward(self, X, state):\n        raise NotImplementedError\n\nclass EncoderDecoder(nn.Module):\n    \"\"\"编码器-解码器架构的基类\"\"\"\n    def __init__(self, encoder, decoder, **kwargs):\n        super(EncoderDecoder, self).__init__(**kwargs)\n        self.encoder = encoder\n        self.decoder = decoder\n\n    def forward(self, enc_X, dec_X, *args):\n        enc_outputs = self.encoder(enc_X, *args)\n        dec_state = self.decoder.init_state(enc_outputs, *args)\n        return self.decoder(dec_X, dec_state)\n",[44,9017,9018,9035,9040,9058,9077,9081,9105,9113,9117,9134,9139,9157,9176,9180,9204,9210,9214,9235,9241,9245,9262,9267,9295,9314,9326,9338,9342,9370,9387,9403],{"__ignoreMap":42},[47,9019,9020,9022,9025,9027,9029,9031,9033],{"class":49,"line":50},[47,9021,3003],{"class":81},[47,9023,9024],{"class":3006}," Encoder",[47,9026,1187],{"class":353},[47,9028,3013],{"class":3012},[47,9030,144],{"class":353},[47,9032,3018],{"class":3012},[47,9034,2805],{"class":353},[47,9036,9037],{"class":49,"line":64},[47,9038,9039],{"class":2810},"    \"\"\"编码器-解码器架构的基本编码器接口\"\"\"\n",[47,9041,9042,9044,9046,9048,9050,9052,9054,9056],{"class":49,"line":74},[47,9043,3030],{"class":81},[47,9045,3033],{"class":118},[47,9047,1187],{"class":353},[47,9049,3039],{"class":3038},[47,9051,2799],{"class":353},[47,9053,3049],{"class":81},[47,9055,3052],{"class":2795},[47,9057,2805],{"class":353},[47,9059,9060,9062,9065,9067,9069,9071,9073,9075],{"class":49,"line":335},[47,9061,3060],{"class":3059},[47,9063,9064],{"class":353},"(Encoder, ",[47,9066,3039],{"class":3066},[47,9068,3069],{"class":353},[47,9070,3072],{"class":118},[47,9072,1187],{"class":353},[47,9074,3049],{"class":81},[47,9076,3079],{"class":353},[47,9078,9079],{"class":49,"line":344},[47,9080,326],{"emptyLinePlaceholder":325},[47,9082,9083,9085,9087,9089,9091,9093,9095,9097,9100,9103],{"class":49,"line":2856},[47,9084,3030],{"class":81},[47,9086,3123],{"class":53},[47,9088,1187],{"class":353},[47,9090,3039],{"class":3038},[47,9092,2799],{"class":353},[47,9094,2796],{"class":2795},[47,9096,2799],{"class":353},[47,9098,9099],{"class":81},"*",[47,9101,9102],{"class":2795},"args",[47,9104,2805],{"class":353},[47,9106,9107,9110],{"class":49,"line":2864},[47,9108,9109],{"class":81},"        raise",[47,9111,9112],{"class":3059}," NotImplementedError\n",[47,9114,9115],{"class":49,"line":2875},[47,9116,326],{"emptyLinePlaceholder":325},[47,9118,9119,9121,9124,9126,9128,9130,9132],{"class":49,"line":2892},[47,9120,3003],{"class":81},[47,9122,9123],{"class":3006}," Decoder",[47,9125,1187],{"class":353},[47,9127,3013],{"class":3012},[47,9129,144],{"class":353},[47,9131,3018],{"class":3012},[47,9133,2805],{"class":353},[47,9135,9136],{"class":49,"line":2908},[47,9137,9138],{"class":2810},"    \"\"\"编码器-解码器架构的基本解码器接口\"\"\"\n",[47,9140,9141,9143,9145,9147,9149,9151,9153,9155],{"class":49,"line":2916},[47,9142,3030],{"class":81},[47,9144,3033],{"class":118},[47,9146,1187],{"class":353},[47,9148,3039],{"class":3038},[47,9150,2799],{"class":353},[47,9152,3049],{"class":81},[47,9154,3052],{"class":2795},[47,9156,2805],{"class":353},[47,9158,9159,9161,9164,9166,9168,9170,9172,9174],{"class":49,"line":2933},[47,9160,3060],{"class":3059},[47,9162,9163],{"class":353},"(Decoder, ",[47,9165,3039],{"class":3066},[47,9167,3069],{"class":353},[47,9169,3072],{"class":118},[47,9171,1187],{"class":353},[47,9173,3049],{"class":81},[47,9175,3079],{"class":353},[47,9177,9178],{"class":49,"line":2939},[47,9179,326],{"emptyLinePlaceholder":325},[47,9181,9182,9184,9187,9189,9191,9193,9196,9198,9200,9202],{"class":49,"line":2964},[47,9183,3030],{"class":81},[47,9185,9186],{"class":53}," init_state",[47,9188,1187],{"class":353},[47,9190,3039],{"class":3038},[47,9192,2799],{"class":353},[47,9194,9195],{"class":2795},"enc_outputs",[47,9197,2799],{"class":353},[47,9199,9099],{"class":81},[47,9201,9102],{"class":2795},[47,9203,2805],{"class":353},[47,9205,9206,9208],{"class":49,"line":2977},[47,9207,9109],{"class":81},[47,9209,9112],{"class":3059},[47,9211,9212],{"class":49,"line":3215},[47,9213,326],{"emptyLinePlaceholder":325},[47,9215,9216,9218,9220,9222,9224,9226,9228,9230,9233],{"class":49,"line":5151},[47,9217,3030],{"class":81},[47,9219,3123],{"class":53},[47,9221,1187],{"class":353},[47,9223,3039],{"class":3038},[47,9225,2799],{"class":353},[47,9227,2796],{"class":2795},[47,9229,2799],{"class":353},[47,9231,9232],{"class":2795},"state",[47,9234,2805],{"class":353},[47,9236,9237,9239],{"class":49,"line":5427},[47,9238,9109],{"class":81},[47,9240,9112],{"class":3059},[47,9242,9243],{"class":49,"line":5433},[47,9244,326],{"emptyLinePlaceholder":325},[47,9246,9247,9249,9252,9254,9256,9258,9260],{"class":49,"line":5439},[47,9248,3003],{"class":81},[47,9250,9251],{"class":3006}," EncoderDecoder",[47,9253,1187],{"class":353},[47,9255,3013],{"class":3012},[47,9257,144],{"class":353},[47,9259,3018],{"class":3012},[47,9261,2805],{"class":353},[47,9263,9264],{"class":49,"line":5445},[47,9265,9266],{"class":2810},"    \"\"\"编码器-解码器架构的基类\"\"\"\n",[47,9268,9269,9271,9273,9275,9277,9279,9282,9284,9287,9289,9291,9293],{"class":49,"line":5466},[47,9270,3030],{"class":81},[47,9272,3033],{"class":118},[47,9274,1187],{"class":353},[47,9276,3039],{"class":3038},[47,9278,2799],{"class":353},[47,9280,9281],{"class":2795},"encoder",[47,9283,2799],{"class":353},[47,9285,9286],{"class":2795},"decoder",[47,9288,2799],{"class":353},[47,9290,3049],{"class":81},[47,9292,3052],{"class":2795},[47,9294,2805],{"class":353},[47,9296,9297,9299,9302,9304,9306,9308,9310,9312],{"class":49,"line":5485},[47,9298,3060],{"class":3059},[47,9300,9301],{"class":353},"(EncoderDecoder, ",[47,9303,3039],{"class":3066},[47,9305,3069],{"class":353},[47,9307,3072],{"class":118},[47,9309,1187],{"class":353},[47,9311,3049],{"class":81},[47,9313,3079],{"class":353},[47,9315,9316,9318,9321,9323],{"class":49,"line":5504},[47,9317,3084],{"class":3066},[47,9319,9320],{"class":353},".encoder ",[47,9322,82],{"class":81},[47,9324,9325],{"class":353}," encoder\n",[47,9327,9328,9330,9333,9335],{"class":49,"line":5509},[47,9329,3084],{"class":3066},[47,9331,9332],{"class":353},".decoder ",[47,9334,82],{"class":81},[47,9336,9337],{"class":353}," decoder\n",[47,9339,9340],{"class":49,"line":5525},[47,9341,326],{"emptyLinePlaceholder":325},[47,9343,9344,9346,9348,9350,9352,9354,9357,9359,9362,9364,9366,9368],{"class":49,"line":5531},[47,9345,3030],{"class":81},[47,9347,3123],{"class":53},[47,9349,1187],{"class":353},[47,9351,3039],{"class":3038},[47,9353,2799],{"class":353},[47,9355,9356],{"class":2795},"enc_X",[47,9358,2799],{"class":353},[47,9360,9361],{"class":2795},"dec_X",[47,9363,2799],{"class":353},[47,9365,9099],{"class":81},[47,9367,9102],{"class":2795},[47,9369,2805],{"class":353},[47,9371,9372,9375,9377,9379,9382,9384],{"class":49,"line":5537},[47,9373,9374],{"class":353},"        enc_outputs ",[47,9376,82],{"class":81},[47,9378,5590],{"class":3066},[47,9380,9381],{"class":353},".encoder(enc_X, ",[47,9383,9099],{"class":81},[47,9385,9386],{"class":353},"args)\n",[47,9388,9389,9392,9394,9396,9399,9401],{"class":49,"line":5547},[47,9390,9391],{"class":353},"        dec_state ",[47,9393,82],{"class":81},[47,9395,5590],{"class":3066},[47,9397,9398],{"class":353},".decoder.init_state(enc_outputs, ",[47,9400,9099],{"class":81},[47,9402,9386],{"class":353},[47,9404,9405,9407,9409],{"class":49,"line":5571},[47,9406,2838],{"class":81},[47,9408,5590],{"class":3066},[47,9410,9411],{"class":353},".decoder(dec_X, dec_state)\n",[10,9413,9415],{"id":9414},"transformer-编码器","transformer 编码器",[14,9417,9418],{},"代码实现如下：",[37,9420,9422],{"className":2778,"code":9421,"language":2780,"meta":42,"style":42},"class EncoderBlock(nn.Module):\n    \"\"\"Transformer编码器块\"\"\"\n    def __init__(self, key_size, query_size, value_size, num_hiddens,\n                 norm_shape, ffn_num_input, ffn_num_hiddens, num_heads,\n                 dropout, use_bias=False, **kwargs):\n        super(EncoderBlock, self).__init__(**kwargs)\n        self.attention = d2l.MultiHeadAttention(\n            key_size, query_size, value_size, num_hiddens, num_heads, dropout,\n            use_bias)\n        self.addnorm1 = AddNorm(norm_shape, dropout)\n        self.ffn = PositionWiseFFN(\n            ffn_num_input, ffn_num_hiddens, num_hiddens)\n        self.addnorm2 = AddNorm(norm_shape, dropout)\n\n    def forward(self, X, valid_lens):\n        Y = self.addnorm1(X, self.attention(X, X, X, valid_lens))\n        return self.addnorm2(Y, self.ffn(Y))\n\nclass TransformerEncoder(Encoder):\n    \"\"\"Transformer编码器\"\"\"\n    def __init__(self, vocab_size, key_size, query_size, value_size,\n                 num_hiddens, norm_shape, ffn_num_input, ffn_num_hiddens,\n                 num_heads, num_layers, dropout, use_bias=False, **kwargs):\n        super(TransformerEncoder, self).__init__(**kwargs)\n        self.num_hiddens = num_hiddens\n        self.embedding = nn.Embedding(vocab_size, num_hiddens)\n        self.pos_encoding = PositionalEncoding(num_hiddens, dropout)\n        self.blks = nn.Sequential()\n        for i in range(num_layers):\n            self.blks.add_module(\"block\"+str(i),\n                EncoderBlock(key_size, query_size, value_size, num_hiddens,\n                             norm_shape, ffn_num_input, ffn_num_hiddens,\n                             num_heads, dropout, use_bias))\n\n    def forward(self, X, valid_lens, *args):\n        # 因为位置编码值在-1和1之间，\n        # 因此嵌入值乘以嵌入维度的平方根进行缩放，\n        # 然后再与位置编码相加。\n        X = self.pos_encoding(self.embedding(X) * math.sqrt(self.num_hiddens))\n        self.attention_weights = [None] * len(self.blks)\n        for i, blk in enumerate(self.blks):\n            X = blk(X, valid_lens)\n            self.attention_weights[i] = blk.attention.attention.attention_weights\n        return X\n",[44,9423,9424,9441,9446,9474,9493,9515,9534,9545,9550,9555,9567,9579,9584,9595,9599,9619,9636,9650,9654,9668,9673,9702,9722,9751,9770,9782,9794,9806,9818,9835,9859,9864,9869,9874,9878,9904,9909,9915,9921,9948,9976,9996,10007,10020],{"__ignoreMap":42},[47,9425,9426,9428,9431,9433,9435,9437,9439],{"class":49,"line":50},[47,9427,3003],{"class":81},[47,9429,9430],{"class":3006}," EncoderBlock",[47,9432,1187],{"class":353},[47,9434,3013],{"class":3012},[47,9436,144],{"class":353},[47,9438,3018],{"class":3012},[47,9440,2805],{"class":353},[47,9442,9443],{"class":49,"line":64},[47,9444,9445],{"class":2810},"    \"\"\"Transformer编码器块\"\"\"\n",[47,9447,9448,9450,9452,9454,9456,9458,9460,9462,9464,9466,9468,9470,9472],{"class":49,"line":74},[47,9449,3030],{"class":81},[47,9451,3033],{"class":118},[47,9453,1187],{"class":353},[47,9455,3039],{"class":3038},[47,9457,2799],{"class":353},[47,9459,5217],{"class":2795},[47,9461,2799],{"class":353},[47,9463,5222],{"class":2795},[47,9465,2799],{"class":353},[47,9467,5227],{"class":2795},[47,9469,2799],{"class":353},[47,9471,5232],{"class":2795},[47,9473,4196],{"class":353},[47,9475,9476,9479,9481,9483,9485,9487,9489,9491],{"class":49,"line":335},[47,9477,9478],{"class":2795},"                 norm_shape",[47,9480,2799],{"class":353},[47,9482,8045],{"class":2795},[47,9484,2799],{"class":353},[47,9486,8050],{"class":2795},[47,9488,2799],{"class":353},[47,9490,4965],{"class":2795},[47,9492,4196],{"class":353},[47,9494,9495,9498,9500,9503,9505,9507,9509,9511,9513],{"class":49,"line":344},[47,9496,9497],{"class":2795},"                 dropout",[47,9499,2799],{"class":353},[47,9501,9502],{"class":2795},"use_bias",[47,9504,82],{"class":81},[47,9506,5253],{"class":118},[47,9508,2799],{"class":353},[47,9510,3049],{"class":81},[47,9512,3052],{"class":2795},[47,9514,2805],{"class":353},[47,9516,9517,9519,9522,9524,9526,9528,9530,9532],{"class":49,"line":2856},[47,9518,3060],{"class":3059},[47,9520,9521],{"class":353},"(EncoderBlock, ",[47,9523,3039],{"class":3066},[47,9525,3069],{"class":353},[47,9527,3072],{"class":118},[47,9529,1187],{"class":353},[47,9531,3049],{"class":81},[47,9533,3079],{"class":353},[47,9535,9536,9538,9540,9542],{"class":49,"line":2864},[47,9537,3084],{"class":3066},[47,9539,5299],{"class":353},[47,9541,82],{"class":81},[47,9543,9544],{"class":353}," d2l.MultiHeadAttention(\n",[47,9546,9547],{"class":49,"line":2875},[47,9548,9549],{"class":353},"            key_size, query_size, value_size, num_hiddens, num_heads, dropout,\n",[47,9551,9552],{"class":49,"line":2892},[47,9553,9554],{"class":353},"            use_bias)\n",[47,9556,9557,9559,9562,9564],{"class":49,"line":2908},[47,9558,3084],{"class":3066},[47,9560,9561],{"class":353},".addnorm1 ",[47,9563,82],{"class":81},[47,9565,9566],{"class":353}," AddNorm(norm_shape, dropout)\n",[47,9568,9569,9571,9574,9576],{"class":49,"line":2916},[47,9570,3084],{"class":3066},[47,9572,9573],{"class":353},".ffn ",[47,9575,82],{"class":81},[47,9577,9578],{"class":353}," PositionWiseFFN(\n",[47,9580,9581],{"class":49,"line":2933},[47,9582,9583],{"class":353},"            ffn_num_input, ffn_num_hiddens, num_hiddens)\n",[47,9585,9586,9588,9591,9593],{"class":49,"line":2939},[47,9587,3084],{"class":3066},[47,9589,9590],{"class":353},".addnorm2 ",[47,9592,82],{"class":81},[47,9594,9566],{"class":353},[47,9596,9597],{"class":49,"line":2964},[47,9598,326],{"emptyLinePlaceholder":325},[47,9600,9601,9603,9605,9607,9609,9611,9613,9615,9617],{"class":49,"line":2977},[47,9602,3030],{"class":81},[47,9604,3123],{"class":53},[47,9606,1187],{"class":353},[47,9608,3039],{"class":3038},[47,9610,2799],{"class":353},[47,9612,2796],{"class":2795},[47,9614,2799],{"class":353},[47,9616,2802],{"class":2795},[47,9618,2805],{"class":353},[47,9620,9621,9624,9626,9628,9631,9633],{"class":49,"line":3215},[47,9622,9623],{"class":353},"        Y ",[47,9625,82],{"class":81},[47,9627,5590],{"class":3066},[47,9629,9630],{"class":353},".addnorm1(X, ",[47,9632,3039],{"class":3066},[47,9634,9635],{"class":353},".attention(X, X, X, valid_lens))\n",[47,9637,9638,9640,9642,9645,9647],{"class":49,"line":5151},[47,9639,2838],{"class":81},[47,9641,5590],{"class":3066},[47,9643,9644],{"class":353},".addnorm2(Y, ",[47,9646,3039],{"class":3066},[47,9648,9649],{"class":353},".ffn(Y))\n",[47,9651,9652],{"class":49,"line":5427},[47,9653,326],{"emptyLinePlaceholder":325},[47,9655,9656,9658,9661,9663,9666],{"class":49,"line":5433},[47,9657,3003],{"class":81},[47,9659,9660],{"class":3006}," TransformerEncoder",[47,9662,1187],{"class":353},[47,9664,9665],{"class":3012},"Encoder",[47,9667,2805],{"class":353},[47,9669,9670],{"class":49,"line":5439},[47,9671,9672],{"class":2810},"    \"\"\"Transformer编码器\"\"\"\n",[47,9674,9675,9677,9679,9681,9683,9685,9688,9690,9692,9694,9696,9698,9700],{"class":49,"line":5445},[47,9676,3030],{"class":81},[47,9678,3033],{"class":118},[47,9680,1187],{"class":353},[47,9682,3039],{"class":3038},[47,9684,2799],{"class":353},[47,9686,9687],{"class":2795},"vocab_size",[47,9689,2799],{"class":353},[47,9691,5217],{"class":2795},[47,9693,2799],{"class":353},[47,9695,5222],{"class":2795},[47,9697,2799],{"class":353},[47,9699,5227],{"class":2795},[47,9701,4196],{"class":353},[47,9703,9704,9707,9709,9712,9714,9716,9718,9720],{"class":49,"line":5466},[47,9705,9706],{"class":2795},"                 num_hiddens",[47,9708,2799],{"class":353},[47,9710,9711],{"class":2795},"norm_shape",[47,9713,2799],{"class":353},[47,9715,8045],{"class":2795},[47,9717,2799],{"class":353},[47,9719,8050],{"class":2795},[47,9721,4196],{"class":353},[47,9723,9724,9726,9728,9731,9733,9735,9737,9739,9741,9743,9745,9747,9749],{"class":49,"line":5485},[47,9725,5239],{"class":2795},[47,9727,2799],{"class":353},[47,9729,9730],{"class":2795},"num_layers",[47,9732,2799],{"class":353},[47,9734,3044],{"class":2795},[47,9736,2799],{"class":353},[47,9738,9502],{"class":2795},[47,9740,82],{"class":81},[47,9742,5253],{"class":118},[47,9744,2799],{"class":353},[47,9746,3049],{"class":81},[47,9748,3052],{"class":2795},[47,9750,2805],{"class":353},[47,9752,9753,9755,9758,9760,9762,9764,9766,9768],{"class":49,"line":5504},[47,9754,3060],{"class":3059},[47,9756,9757],{"class":353},"(TransformerEncoder, ",[47,9759,3039],{"class":3066},[47,9761,3069],{"class":353},[47,9763,3072],{"class":118},[47,9765,1187],{"class":353},[47,9767,3049],{"class":81},[47,9769,3079],{"class":353},[47,9771,9772,9774,9777,9779],{"class":49,"line":5509},[47,9773,3084],{"class":3066},[47,9775,9776],{"class":353},".num_hiddens ",[47,9778,82],{"class":81},[47,9780,9781],{"class":353}," num_hiddens\n",[47,9783,9784,9786,9789,9791],{"class":49,"line":5525},[47,9785,3084],{"class":3066},[47,9787,9788],{"class":353},".embedding ",[47,9790,82],{"class":81},[47,9792,9793],{"class":353}," nn.Embedding(vocab_size, num_hiddens)\n",[47,9795,9796,9798,9801,9803],{"class":49,"line":5531},[47,9797,3084],{"class":3066},[47,9799,9800],{"class":353},".pos_encoding ",[47,9802,82],{"class":81},[47,9804,9805],{"class":353}," PositionalEncoding(num_hiddens, dropout)\n",[47,9807,9808,9810,9813,9815],{"class":49,"line":5537},[47,9809,3084],{"class":3066},[47,9811,9812],{"class":353},".blks ",[47,9814,82],{"class":81},[47,9816,9817],{"class":353}," nn.Sequential()\n",[47,9819,9820,9823,9826,9829,9832],{"class":49,"line":5547},[47,9821,9822],{"class":81},"        for",[47,9824,9825],{"class":353}," i ",[47,9827,9828],{"class":81},"in",[47,9830,9831],{"class":67}," range",[47,9833,9834],{"class":353},"(num_layers):\n",[47,9836,9837,9840,9843,9846,9849,9851,9853,9856],{"class":49,"line":5571},[47,9838,9839],{"class":3066},"            self",[47,9841,9842],{"class":353},".blks.add_module(",[47,9844,9845],{"class":125},"\"",[47,9847,9848],{"class":57},"block",[47,9850,9845],{"class":125},[47,9852,6147],{"class":81},[47,9854,9855],{"class":3059},"str",[47,9857,9858],{"class":353},"(i),\n",[47,9860,9861],{"class":49,"line":5576},[47,9862,9863],{"class":353},"                EncoderBlock(key_size, query_size, value_size, num_hiddens,\n",[47,9865,9866],{"class":49,"line":5582},[47,9867,9868],{"class":353},"                             norm_shape, ffn_num_input, ffn_num_hiddens,\n",[47,9870,9871],{"class":49,"line":5596},[47,9872,9873],{"class":353},"                             num_heads, dropout, use_bias))\n",[47,9875,9876],{"class":49,"line":5601},[47,9877,326],{"emptyLinePlaceholder":325},[47,9879,9880,9882,9884,9886,9888,9890,9892,9894,9896,9898,9900,9902],{"class":49,"line":5607},[47,9881,3030],{"class":81},[47,9883,3123],{"class":53},[47,9885,1187],{"class":353},[47,9887,3039],{"class":3038},[47,9889,2799],{"class":353},[47,9891,2796],{"class":2795},[47,9893,2799],{"class":353},[47,9895,2802],{"class":2795},[47,9897,2799],{"class":353},[47,9899,9099],{"class":81},[47,9901,9102],{"class":2795},[47,9903,2805],{"class":353},[47,9905,9906],{"class":49,"line":5622},[47,9907,9908],{"class":331},"        # 因为位置编码值在-1和1之间，\n",[47,9910,9912],{"class":49,"line":9911},37,[47,9913,9914],{"class":331},"        # 因此嵌入值乘以嵌入维度的平方根进行缩放，\n",[47,9916,9918],{"class":49,"line":9917},38,[47,9919,9920],{"class":331},"        # 然后再与位置编码相加。\n",[47,9922,9924,9926,9928,9930,9933,9935,9938,9940,9943,9945],{"class":49,"line":9923},39,[47,9925,2942],{"class":353},[47,9927,82],{"class":81},[47,9929,5590],{"class":3066},[47,9931,9932],{"class":353},".pos_encoding(",[47,9934,3039],{"class":3066},[47,9936,9937],{"class":353},".embedding(X) ",[47,9939,9099],{"class":81},[47,9941,9942],{"class":353}," math.sqrt(",[47,9944,3039],{"class":3066},[47,9946,9947],{"class":353},".num_hiddens))\n",[47,9949,9951,9953,9955,9957,9960,9962,9964,9966,9969,9971,9973],{"class":49,"line":9950},40,[47,9952,3084],{"class":3066},[47,9954,3207],{"class":353},[47,9956,82],{"class":81},[47,9958,9959],{"class":353}," [",[47,9961,3151],{"class":118},[47,9963,7570],{"class":353},[47,9965,9099],{"class":81},[47,9967,9968],{"class":67}," len",[47,9970,1187],{"class":353},[47,9972,3039],{"class":3066},[47,9974,9975],{"class":353},".blks)\n",[47,9977,9979,9981,9984,9986,9989,9991,9993],{"class":49,"line":9978},41,[47,9980,9822],{"class":81},[47,9982,9983],{"class":353}," i, blk ",[47,9985,9828],{"class":81},[47,9987,9988],{"class":67}," enumerate",[47,9990,1187],{"class":353},[47,9992,3039],{"class":3066},[47,9994,9995],{"class":353},".blks):\n",[47,9997,9999,10002,10004],{"class":49,"line":9998},42,[47,10000,10001],{"class":353},"            X ",[47,10003,82],{"class":81},[47,10005,10006],{"class":353}," blk(X, valid_lens)\n",[47,10008,10010,10012,10015,10017],{"class":49,"line":10009},43,[47,10011,9839],{"class":3066},[47,10013,10014],{"class":353},".attention_weights[i] ",[47,10016,82],{"class":81},[47,10018,10019],{"class":353}," blk.attention.attention.attention_weights\n",[47,10021,10023,10025],{"class":49,"line":10022},44,[47,10024,2838],{"class":81},[47,10026,10027],{"class":353}," X\n",[10,10029,10031],{"id":10030},"transformer-解码器","transformer 解码器",[14,10033,10034],{},"代码如下：",[37,10036,10038],{"className":2778,"code":10037,"language":2780,"meta":42,"style":42},"class DecoderBlock(nn.Module):\n    \"\"\"解码器中第i个块\"\"\"\n    def __init__(self, key_size, query_size, value_size, num_hiddens,\n                 norm_shape, ffn_num_input, ffn_num_hiddens, num_heads,\n                 dropout, i, **kwargs):\n        super(DecoderBlock, self).__init__(**kwargs)\n        self.i = i\n        self.attention1 = d2l.MultiHeadAttention(\n            key_size, query_size, value_size, num_hiddens, num_heads, dropout)\n        self.addnorm1 = AddNorm(norm_shape, dropout)\n        self.attention2 = d2l.MultiHeadAttention(\n            key_size, query_size, value_size, num_hiddens, num_heads, dropout)\n        self.addnorm2 = AddNorm(norm_shape, dropout)\n        self.ffn = PositionWiseFFN(ffn_num_input, ffn_num_hiddens,\n                                   num_hiddens)\n        self.addnorm3 = AddNorm(norm_shape, dropout)\n\n    def forward(self, X, state):\n        enc_outputs, enc_valid_lens = state[0], state[1]\n        # 训练阶段，输出序列的所有词元都在同一时间处理，\n        # 因此state[2][self.i]初始化为None。\n        # 预测阶段，输出序列是通过词元一个接着一个解码的，\n        # 因此state[2][self.i]包含着直到当前时间步第i个块解码的输出表示\n        if state[2][self.i] is None:\n            key_values = X\n        else:\n            key_values = torch.cat((state[2][self.i], X), axis=1)\n        state[2][self.i] = key_values\n        if self.training:\n            batch_size, num_steps, _ = X.shape\n            # dec_valid_lens的开头:(batch_size,num_steps),\n            # 其中每一行是[1,2,...,num_steps]\n            dec_valid_lens = torch.arange(\n                1, num_steps + 1, device=X.device).repeat(batch_size, 1)\n        else:\n            dec_valid_lens = None\n\n        # 自注意力\n        X2 = self.attention1(X, key_values, key_values, dec_valid_lens)\n        Y = self.addnorm1(X, X2)\n        # 编码器－解码器注意力。\n        # enc_outputs的开头:(batch_size,num_steps,num_hiddens)\n        Y2 = self.attention2(Y, enc_outputs, enc_outputs, enc_valid_lens)\n        Z = self.addnorm2(Y, Y2)\n        return self.addnorm3(Z, self.ffn(Z)), state\n\nclass TransformerDecoder(AttentionDecoder):\n    def __init__(self, vocab_size, key_size, query_size, value_size,\n                 num_hiddens, norm_shape, ffn_num_input, ffn_num_hiddens,\n                 num_heads, num_layers, dropout, **kwargs):\n        super(TransformerDecoder, self).__init__(**kwargs)\n        self.num_hiddens = num_hiddens\n        self.num_layers = num_layers\n        self.embedding = nn.Embedding(vocab_size, num_hiddens)\n        self.pos_encoding = PositionalEncoding(num_hiddens, dropout)\n        self.blks = nn.Sequential()\n        for i in range(num_layers):\n            self.blks.add_module(\"block\"+str(i),\n                DecoderBlock(key_size, query_size, value_size, num_hiddens,\n                             norm_shape, ffn_num_input, ffn_num_hiddens,\n                             num_heads, dropout, i))\n        self.dense = nn.Linear(num_hiddens, vocab_size)\n\n    def init_state(self, enc_outputs, enc_valid_lens, *args):\n        return [enc_outputs, enc_valid_lens, [None] * self.num_layers]\n\n    def forward(self, X, state):\n        X = self.pos_encoding(self.embedding(X) * math.sqrt(self.num_hiddens))\n        self._attention_weights = [[None] * len(self.blks) for _ in range (2)]\n        for i, blk in enumerate(self.blks):\n            X, state = blk(X, state)\n            # 解码器自注意力权重\n            self._attention_weights[0][i] = blk.attention1.attention.attention_weights\n            # “编码器－解码器”自注意力权重\n            self._attention_weights[1][i] = blk.attention2.attention.attention_weights\n        return self.dense(X), state\n\n    @property\n    def attention_weights(self):\n        return self._attention_weights\n",[44,10039,10040,10057,10062,10090,10108,10124,10143,10155,10166,10171,10181,10192,10196,10206,10217,10222,10233,10237,10257,10276,10281,10286,10291,10296,10318,10327,10333,10360,10378,10387,10396,10401,10406,10416,10442,10448,10457,10461,10466,10478,10489,10494,10499,10511,10523,10538,10543,10558,10587,10606,10627,10647,10658,10671,10682,10693,10704,10717,10736,10742,10747,10753,10766,10771,10799,10818,10823,10844,10867,10913,10930,10941,10947,10965,10971,10987,10997,11002,11011,11025],{"__ignoreMap":42},[47,10041,10042,10044,10047,10049,10051,10053,10055],{"class":49,"line":50},[47,10043,3003],{"class":81},[47,10045,10046],{"class":3006}," DecoderBlock",[47,10048,1187],{"class":353},[47,10050,3013],{"class":3012},[47,10052,144],{"class":353},[47,10054,3018],{"class":3012},[47,10056,2805],{"class":353},[47,10058,10059],{"class":49,"line":64},[47,10060,10061],{"class":2810},"    \"\"\"解码器中第i个块\"\"\"\n",[47,10063,10064,10066,10068,10070,10072,10074,10076,10078,10080,10082,10084,10086,10088],{"class":49,"line":74},[47,10065,3030],{"class":81},[47,10067,3033],{"class":118},[47,10069,1187],{"class":353},[47,10071,3039],{"class":3038},[47,10073,2799],{"class":353},[47,10075,5217],{"class":2795},[47,10077,2799],{"class":353},[47,10079,5222],{"class":2795},[47,10081,2799],{"class":353},[47,10083,5227],{"class":2795},[47,10085,2799],{"class":353},[47,10087,5232],{"class":2795},[47,10089,4196],{"class":353},[47,10091,10092,10094,10096,10098,10100,10102,10104,10106],{"class":49,"line":335},[47,10093,9478],{"class":2795},[47,10095,2799],{"class":353},[47,10097,8045],{"class":2795},[47,10099,2799],{"class":353},[47,10101,8050],{"class":2795},[47,10103,2799],{"class":353},[47,10105,4965],{"class":2795},[47,10107,4196],{"class":353},[47,10109,10110,10112,10114,10116,10118,10120,10122],{"class":49,"line":344},[47,10111,9497],{"class":2795},[47,10113,2799],{"class":353},[47,10115,1178],{"class":2795},[47,10117,2799],{"class":353},[47,10119,3049],{"class":81},[47,10121,3052],{"class":2795},[47,10123,2805],{"class":353},[47,10125,10126,10128,10131,10133,10135,10137,10139,10141],{"class":49,"line":2856},[47,10127,3060],{"class":3059},[47,10129,10130],{"class":353},"(DecoderBlock, ",[47,10132,3039],{"class":3066},[47,10134,3069],{"class":353},[47,10136,3072],{"class":118},[47,10138,1187],{"class":353},[47,10140,3049],{"class":81},[47,10142,3079],{"class":353},[47,10144,10145,10147,10150,10152],{"class":49,"line":2864},[47,10146,3084],{"class":3066},[47,10148,10149],{"class":353},".i ",[47,10151,82],{"class":81},[47,10153,10154],{"class":353}," i\n",[47,10156,10157,10159,10162,10164],{"class":49,"line":2875},[47,10158,3084],{"class":3066},[47,10160,10161],{"class":353},".attention1 ",[47,10163,82],{"class":81},[47,10165,9544],{"class":353},[47,10167,10168],{"class":49,"line":2892},[47,10169,10170],{"class":353},"            key_size, query_size, value_size, num_hiddens, num_heads, dropout)\n",[47,10172,10173,10175,10177,10179],{"class":49,"line":2908},[47,10174,3084],{"class":3066},[47,10176,9561],{"class":353},[47,10178,82],{"class":81},[47,10180,9566],{"class":353},[47,10182,10183,10185,10188,10190],{"class":49,"line":2916},[47,10184,3084],{"class":3066},[47,10186,10187],{"class":353},".attention2 ",[47,10189,82],{"class":81},[47,10191,9544],{"class":353},[47,10193,10194],{"class":49,"line":2933},[47,10195,10170],{"class":353},[47,10197,10198,10200,10202,10204],{"class":49,"line":2939},[47,10199,3084],{"class":3066},[47,10201,9590],{"class":353},[47,10203,82],{"class":81},[47,10205,9566],{"class":353},[47,10207,10208,10210,10212,10214],{"class":49,"line":2964},[47,10209,3084],{"class":3066},[47,10211,9573],{"class":353},[47,10213,82],{"class":81},[47,10215,10216],{"class":353}," PositionWiseFFN(ffn_num_input, ffn_num_hiddens,\n",[47,10218,10219],{"class":49,"line":2977},[47,10220,10221],{"class":353},"                                   num_hiddens)\n",[47,10223,10224,10226,10229,10231],{"class":49,"line":3215},[47,10225,3084],{"class":3066},[47,10227,10228],{"class":353},".addnorm3 ",[47,10230,82],{"class":81},[47,10232,9566],{"class":353},[47,10234,10235],{"class":49,"line":5151},[47,10236,326],{"emptyLinePlaceholder":325},[47,10238,10239,10241,10243,10245,10247,10249,10251,10253,10255],{"class":49,"line":5427},[47,10240,3030],{"class":81},[47,10242,3123],{"class":53},[47,10244,1187],{"class":353},[47,10246,3039],{"class":3038},[47,10248,2799],{"class":353},[47,10250,2796],{"class":2795},[47,10252,2799],{"class":353},[47,10254,9232],{"class":2795},[47,10256,2805],{"class":353},[47,10258,10259,10262,10264,10267,10269,10272,10274],{"class":49,"line":5433},[47,10260,10261],{"class":353},"        enc_outputs, enc_valid_lens ",[47,10263,82],{"class":81},[47,10265,10266],{"class":353}," state[",[47,10268,2141],{"class":118},[47,10270,10271],{"class":353},"], state[",[47,10273,472],{"class":118},[47,10275,3170],{"class":353},[47,10277,10278],{"class":49,"line":5439},[47,10279,10280],{"class":331},"        # 训练阶段，输出序列的所有词元都在同一时间处理，\n",[47,10282,10283],{"class":49,"line":5445},[47,10284,10285],{"class":331},"        # 因此state[2][self.i]初始化为None。\n",[47,10287,10288],{"class":49,"line":5466},[47,10289,10290],{"class":331},"        # 预测阶段，输出序列是通过词元一个接着一个解码的，\n",[47,10292,10293],{"class":49,"line":5485},[47,10294,10295],{"class":331},"        # 因此state[2][self.i]包含着直到当前时间步第i个块解码的输出表示\n",[47,10297,10298,10300,10302,10304,10307,10309,10312,10314,10316],{"class":49,"line":5504},[47,10299,2878],{"class":81},[47,10301,10266],{"class":353},[47,10303,3192],{"class":118},[47,10305,10306],{"class":353},"][",[47,10308,3039],{"class":3066},[47,10310,10311],{"class":353},".i] ",[47,10313,2827],{"class":81},[47,10315,2830],{"class":118},[47,10317,2833],{"class":353},[47,10319,10320,10323,10325],{"class":49,"line":5509},[47,10321,10322],{"class":353},"            key_values ",[47,10324,82],{"class":81},[47,10326,10027],{"class":353},[47,10328,10329,10331],{"class":49,"line":5525},[47,10330,2911],{"class":81},[47,10332,2833],{"class":353},[47,10334,10335,10337,10339,10342,10344,10346,10348,10351,10354,10356,10358],{"class":49,"line":5531},[47,10336,10322],{"class":353},[47,10338,82],{"class":81},[47,10340,10341],{"class":353}," torch.cat((state[",[47,10343,3192],{"class":118},[47,10345,10306],{"class":353},[47,10347,3039],{"class":3066},[47,10349,10350],{"class":353},".i], X), ",[47,10352,10353],{"class":2844},"axis",[47,10355,82],{"class":81},[47,10357,472],{"class":118},[47,10359,2853],{"class":353},[47,10361,10362,10365,10367,10369,10371,10373,10375],{"class":49,"line":5537},[47,10363,10364],{"class":353},"        state[",[47,10366,3192],{"class":118},[47,10368,10306],{"class":353},[47,10370,3039],{"class":3066},[47,10372,10311],{"class":353},[47,10374,82],{"class":81},[47,10376,10377],{"class":353}," key_values\n",[47,10379,10380,10382,10384],{"class":49,"line":5547},[47,10381,2878],{"class":81},[47,10383,5590],{"class":3066},[47,10385,10386],{"class":353},".training:\n",[47,10388,10389,10392,10394],{"class":49,"line":5571},[47,10390,10391],{"class":353},"            batch_size, num_steps, _ ",[47,10393,82],{"class":81},[47,10395,2872],{"class":353},[47,10397,10398],{"class":49,"line":5576},[47,10399,10400],{"class":331},"            # dec_valid_lens的开头:(batch_size,num_steps),\n",[47,10402,10403],{"class":49,"line":5582},[47,10404,10405],{"class":331},"            # 其中每一行是[1,2,...,num_steps]\n",[47,10407,10408,10411,10413],{"class":49,"line":5596},[47,10409,10410],{"class":353},"            dec_valid_lens ",[47,10412,82],{"class":81},[47,10414,10415],{"class":353}," torch.arange(\n",[47,10417,10418,10421,10424,10426,10428,10430,10433,10435,10438,10440],{"class":49,"line":5601},[47,10419,10420],{"class":118},"                1",[47,10422,10423],{"class":353},", num_steps ",[47,10425,6147],{"class":81},[47,10427,2887],{"class":118},[47,10429,2799],{"class":353},[47,10431,10432],{"class":2844},"device",[47,10434,82],{"class":81},[47,10436,10437],{"class":353},"X.device).repeat(batch_size, ",[47,10439,472],{"class":118},[47,10441,2853],{"class":353},[47,10443,10444,10446],{"class":49,"line":5607},[47,10445,2911],{"class":81},[47,10447,2833],{"class":353},[47,10449,10450,10452,10454],{"class":49,"line":5622},[47,10451,10410],{"class":353},[47,10453,82],{"class":81},[47,10455,10456],{"class":118}," None\n",[47,10458,10459],{"class":49,"line":9911},[47,10460,326],{"emptyLinePlaceholder":325},[47,10462,10463],{"class":49,"line":9917},[47,10464,10465],{"class":331},"        # 自注意力\n",[47,10467,10468,10471,10473,10475],{"class":49,"line":9923},[47,10469,10470],{"class":353},"        X2 ",[47,10472,82],{"class":81},[47,10474,5590],{"class":3066},[47,10476,10477],{"class":353},".attention1(X, key_values, key_values, dec_valid_lens)\n",[47,10479,10480,10482,10484,10486],{"class":49,"line":9950},[47,10481,9623],{"class":353},[47,10483,82],{"class":81},[47,10485,5590],{"class":3066},[47,10487,10488],{"class":353},".addnorm1(X, X2)\n",[47,10490,10491],{"class":49,"line":9978},[47,10492,10493],{"class":331},"        # 编码器－解码器注意力。\n",[47,10495,10496],{"class":49,"line":9998},[47,10497,10498],{"class":331},"        # enc_outputs的开头:(batch_size,num_steps,num_hiddens)\n",[47,10500,10501,10504,10506,10508],{"class":49,"line":10009},[47,10502,10503],{"class":353},"        Y2 ",[47,10505,82],{"class":81},[47,10507,5590],{"class":3066},[47,10509,10510],{"class":353},".attention2(Y, enc_outputs, enc_outputs, enc_valid_lens)\n",[47,10512,10513,10516,10518,10520],{"class":49,"line":10022},[47,10514,10515],{"class":353},"        Z ",[47,10517,82],{"class":81},[47,10519,5590],{"class":3066},[47,10521,10522],{"class":353},".addnorm2(Y, Y2)\n",[47,10524,10526,10528,10530,10533,10535],{"class":49,"line":10525},45,[47,10527,2838],{"class":81},[47,10529,5590],{"class":3066},[47,10531,10532],{"class":353},".addnorm3(Z, ",[47,10534,3039],{"class":3066},[47,10536,10537],{"class":353},".ffn(Z)), state\n",[47,10539,10541],{"class":49,"line":10540},46,[47,10542,326],{"emptyLinePlaceholder":325},[47,10544,10546,10548,10551,10553,10556],{"class":49,"line":10545},47,[47,10547,3003],{"class":81},[47,10549,10550],{"class":3006}," TransformerDecoder",[47,10552,1187],{"class":353},[47,10554,10555],{"class":3012},"AttentionDecoder",[47,10557,2805],{"class":353},[47,10559,10561,10563,10565,10567,10569,10571,10573,10575,10577,10579,10581,10583,10585],{"class":49,"line":10560},48,[47,10562,3030],{"class":81},[47,10564,3033],{"class":118},[47,10566,1187],{"class":353},[47,10568,3039],{"class":3038},[47,10570,2799],{"class":353},[47,10572,9687],{"class":2795},[47,10574,2799],{"class":353},[47,10576,5217],{"class":2795},[47,10578,2799],{"class":353},[47,10580,5222],{"class":2795},[47,10582,2799],{"class":353},[47,10584,5227],{"class":2795},[47,10586,4196],{"class":353},[47,10588,10590,10592,10594,10596,10598,10600,10602,10604],{"class":49,"line":10589},49,[47,10591,9706],{"class":2795},[47,10593,2799],{"class":353},[47,10595,9711],{"class":2795},[47,10597,2799],{"class":353},[47,10599,8045],{"class":2795},[47,10601,2799],{"class":353},[47,10603,8050],{"class":2795},[47,10605,4196],{"class":353},[47,10607,10609,10611,10613,10615,10617,10619,10621,10623,10625],{"class":49,"line":10608},50,[47,10610,5239],{"class":2795},[47,10612,2799],{"class":353},[47,10614,9730],{"class":2795},[47,10616,2799],{"class":353},[47,10618,3044],{"class":2795},[47,10620,2799],{"class":353},[47,10622,3049],{"class":81},[47,10624,3052],{"class":2795},[47,10626,2805],{"class":353},[47,10628,10630,10632,10635,10637,10639,10641,10643,10645],{"class":49,"line":10629},51,[47,10631,3060],{"class":3059},[47,10633,10634],{"class":353},"(TransformerDecoder, ",[47,10636,3039],{"class":3066},[47,10638,3069],{"class":353},[47,10640,3072],{"class":118},[47,10642,1187],{"class":353},[47,10644,3049],{"class":81},[47,10646,3079],{"class":353},[47,10648,10650,10652,10654,10656],{"class":49,"line":10649},52,[47,10651,3084],{"class":3066},[47,10653,9776],{"class":353},[47,10655,82],{"class":81},[47,10657,9781],{"class":353},[47,10659,10661,10663,10666,10668],{"class":49,"line":10660},53,[47,10662,3084],{"class":3066},[47,10664,10665],{"class":353},".num_layers ",[47,10667,82],{"class":81},[47,10669,10670],{"class":353}," num_layers\n",[47,10672,10674,10676,10678,10680],{"class":49,"line":10673},54,[47,10675,3084],{"class":3066},[47,10677,9788],{"class":353},[47,10679,82],{"class":81},[47,10681,9793],{"class":353},[47,10683,10685,10687,10689,10691],{"class":49,"line":10684},55,[47,10686,3084],{"class":3066},[47,10688,9800],{"class":353},[47,10690,82],{"class":81},[47,10692,9805],{"class":353},[47,10694,10696,10698,10700,10702],{"class":49,"line":10695},56,[47,10697,3084],{"class":3066},[47,10699,9812],{"class":353},[47,10701,82],{"class":81},[47,10703,9817],{"class":353},[47,10705,10707,10709,10711,10713,10715],{"class":49,"line":10706},57,[47,10708,9822],{"class":81},[47,10710,9825],{"class":353},[47,10712,9828],{"class":81},[47,10714,9831],{"class":67},[47,10716,9834],{"class":353},[47,10718,10720,10722,10724,10726,10728,10730,10732,10734],{"class":49,"line":10719},58,[47,10721,9839],{"class":3066},[47,10723,9842],{"class":353},[47,10725,9845],{"class":125},[47,10727,9848],{"class":57},[47,10729,9845],{"class":125},[47,10731,6147],{"class":81},[47,10733,9855],{"class":3059},[47,10735,9858],{"class":353},[47,10737,10739],{"class":49,"line":10738},59,[47,10740,10741],{"class":353},"                DecoderBlock(key_size, query_size, value_size, num_hiddens,\n",[47,10743,10745],{"class":49,"line":10744},60,[47,10746,9868],{"class":353},[47,10748,10750],{"class":49,"line":10749},61,[47,10751,10752],{"class":353},"                             num_heads, dropout, i))\n",[47,10754,10756,10758,10761,10763],{"class":49,"line":10755},62,[47,10757,3084],{"class":3066},[47,10759,10760],{"class":353},".dense ",[47,10762,82],{"class":81},[47,10764,10765],{"class":353}," nn.Linear(num_hiddens, vocab_size)\n",[47,10767,10769],{"class":49,"line":10768},63,[47,10770,326],{"emptyLinePlaceholder":325},[47,10772,10774,10776,10778,10780,10782,10784,10786,10788,10791,10793,10795,10797],{"class":49,"line":10773},64,[47,10775,3030],{"class":81},[47,10777,9186],{"class":53},[47,10779,1187],{"class":353},[47,10781,3039],{"class":3038},[47,10783,2799],{"class":353},[47,10785,9195],{"class":2795},[47,10787,2799],{"class":353},[47,10789,10790],{"class":2795},"enc_valid_lens",[47,10792,2799],{"class":353},[47,10794,9099],{"class":81},[47,10796,9102],{"class":2795},[47,10798,2805],{"class":353},[47,10800,10802,10804,10807,10809,10811,10813,10815],{"class":49,"line":10801},65,[47,10803,2838],{"class":81},[47,10805,10806],{"class":353}," [enc_outputs, enc_valid_lens, [",[47,10808,3151],{"class":118},[47,10810,7570],{"class":353},[47,10812,9099],{"class":81},[47,10814,5590],{"class":3066},[47,10816,10817],{"class":353},".num_layers]\n",[47,10819,10821],{"class":49,"line":10820},66,[47,10822,326],{"emptyLinePlaceholder":325},[47,10824,10826,10828,10830,10832,10834,10836,10838,10840,10842],{"class":49,"line":10825},67,[47,10827,3030],{"class":81},[47,10829,3123],{"class":53},[47,10831,1187],{"class":353},[47,10833,3039],{"class":3038},[47,10835,2799],{"class":353},[47,10837,2796],{"class":2795},[47,10839,2799],{"class":353},[47,10841,9232],{"class":2795},[47,10843,2805],{"class":353},[47,10845,10847,10849,10851,10853,10855,10857,10859,10861,10863,10865],{"class":49,"line":10846},68,[47,10848,2942],{"class":353},[47,10850,82],{"class":81},[47,10852,5590],{"class":3066},[47,10854,9932],{"class":353},[47,10856,3039],{"class":3066},[47,10858,9937],{"class":353},[47,10860,9099],{"class":81},[47,10862,9942],{"class":353},[47,10864,3039],{"class":3066},[47,10866,9947],{"class":353},[47,10868,10870,10872,10875,10877,10880,10882,10884,10886,10888,10890,10892,10895,10898,10901,10903,10905,10908,10910],{"class":49,"line":10869},69,[47,10871,3084],{"class":3066},[47,10873,10874],{"class":353},"._attention_weights ",[47,10876,82],{"class":81},[47,10878,10879],{"class":353}," [[",[47,10881,3151],{"class":118},[47,10883,7570],{"class":353},[47,10885,9099],{"class":81},[47,10887,9968],{"class":67},[47,10889,1187],{"class":353},[47,10891,3039],{"class":3066},[47,10893,10894],{"class":353},".blks) ",[47,10896,10897],{"class":81},"for",[47,10899,10900],{"class":353}," _ ",[47,10902,9828],{"class":81},[47,10904,9831],{"class":67},[47,10906,10907],{"class":353}," (",[47,10909,3192],{"class":118},[47,10911,10912],{"class":353},")]\n",[47,10914,10916,10918,10920,10922,10924,10926,10928],{"class":49,"line":10915},70,[47,10917,9822],{"class":81},[47,10919,9983],{"class":353},[47,10921,9828],{"class":81},[47,10923,9988],{"class":67},[47,10925,1187],{"class":353},[47,10927,3039],{"class":3066},[47,10929,9995],{"class":353},[47,10931,10933,10936,10938],{"class":49,"line":10932},71,[47,10934,10935],{"class":353},"            X, state ",[47,10937,82],{"class":81},[47,10939,10940],{"class":353}," blk(X, state)\n",[47,10942,10944],{"class":49,"line":10943},72,[47,10945,10946],{"class":331},"            # 解码器自注意力权重\n",[47,10948,10950,10952,10955,10957,10960,10962],{"class":49,"line":10949},73,[47,10951,9839],{"class":3066},[47,10953,10954],{"class":353},"._attention_weights[",[47,10956,2141],{"class":118},[47,10958,10959],{"class":353},"][i] ",[47,10961,82],{"class":81},[47,10963,10964],{"class":353}," blk.attention1.attention.attention_weights\n",[47,10966,10968],{"class":49,"line":10967},74,[47,10969,10970],{"class":331},"            # “编码器－解码器”自注意力权重\n",[47,10972,10974,10976,10978,10980,10982,10984],{"class":49,"line":10973},75,[47,10975,9839],{"class":3066},[47,10977,10954],{"class":353},[47,10979,472],{"class":118},[47,10981,10959],{"class":353},[47,10983,82],{"class":81},[47,10985,10986],{"class":353}," blk.attention2.attention.attention_weights\n",[47,10988,10990,10992,10994],{"class":49,"line":10989},76,[47,10991,2838],{"class":81},[47,10993,5590],{"class":3066},[47,10995,10996],{"class":353},".dense(X), state\n",[47,10998,11000],{"class":49,"line":10999},77,[47,11001,326],{"emptyLinePlaceholder":325},[47,11003,11005,11008],{"class":49,"line":11004},78,[47,11006,11007],{"class":53},"    @",[47,11009,11010],{"class":3059},"property\n",[47,11012,11014,11016,11019,11021,11023],{"class":49,"line":11013},79,[47,11015,3030],{"class":81},[47,11017,11018],{"class":53}," attention_weights",[47,11020,1187],{"class":353},[47,11022,3039],{"class":3038},[47,11024,2805],{"class":353},[47,11026,11028,11030,11032],{"class":49,"line":11027},80,[47,11029,2838],{"class":81},[47,11031,5590],{"class":3066},[47,11033,11034],{"class":353},"._attention_weights\n",[10,11036,11037],{"id":11037},"训练",[14,11039,11040],{},"同样，训练时需要屏蔽部分不需要的内容：",[37,11042,11044],{"className":2778,"code":11043,"language":2780,"meta":42,"style":42},"def sequence_mask(X, valid_len, value=0):\n    \"\"\"在序列中屏蔽不相关的项\"\"\"\n    maxlen = X.size(1)\n    mask = torch.arange((maxlen), dtype=torch.float32,\n                        device=X.device)[None, :] \u003C valid_len[:, None]\n    X[~mask] = value\n    return X\n\nclass MaskedSoftmaxCELoss(nn.CrossEntropyLoss):\n    \"\"\"带遮蔽的softmax交叉熵损失函数\"\"\"\n    # pred的形状：(batch_size,num_steps,vocab_size)\n    # label的形状：(batch_size,num_steps)\n    # valid_len的形状：(batch_size,)\n    def forward(self, pred, label, valid_len):\n        weights = torch.ones_like(label)\n        weights = sequence_mask(weights, valid_len)\n        self.reduction='none'\n        unweighted_loss = super(MaskedSoftmaxCELoss, self).forward(\n            pred.permute(0, 2, 1), label)\n        weighted_loss = (unweighted_loss * weights).mean(dim=1)\n        return weighted_loss\n",[44,11045,11046,11073,11078,11092,11110,11139,11155,11161,11165,11183,11188,11193,11198,11203,11229,11239,11248,11265,11283,11301,11324],{"__ignoreMap":42},[47,11047,11048,11050,11053,11055,11057,11059,11062,11064,11067,11069,11071],{"class":49,"line":50},[47,11049,2787],{"class":81},[47,11051,11052],{"class":53}," sequence_mask",[47,11054,1187],{"class":353},[47,11056,2796],{"class":2795},[47,11058,2799],{"class":353},[47,11060,11061],{"class":2795},"valid_len",[47,11063,2799],{"class":353},[47,11065,11066],{"class":2795},"value",[47,11068,82],{"class":81},[47,11070,2141],{"class":118},[47,11072,2805],{"class":353},[47,11074,11075],{"class":49,"line":64},[47,11076,11077],{"class":2810},"    \"\"\"在序列中屏蔽不相关的项\"\"\"\n",[47,11079,11080,11083,11085,11088,11090],{"class":49,"line":74},[47,11081,11082],{"class":353},"    maxlen ",[47,11084,82],{"class":81},[47,11086,11087],{"class":353}," X.size(",[47,11089,472],{"class":118},[47,11091,2853],{"class":353},[47,11093,11094,11097,11099,11102,11105,11107],{"class":49,"line":335},[47,11095,11096],{"class":353},"    mask ",[47,11098,82],{"class":81},[47,11100,11101],{"class":353}," torch.arange((maxlen), ",[47,11103,11104],{"class":2844},"dtype",[47,11106,82],{"class":81},[47,11108,11109],{"class":353},"torch.float32,\n",[47,11111,11112,11115,11117,11120,11122,11125,11128,11131,11133,11135,11137],{"class":49,"line":344},[47,11113,11114],{"class":2844},"                        device",[47,11116,82],{"class":81},[47,11118,11119],{"class":353},"X.device)[",[47,11121,3151],{"class":118},[47,11123,11124],{"class":353},", :] ",[47,11126,11127],{"class":81},"\u003C",[47,11129,11130],{"class":353}," valid_len[",[47,11132,7554],{"class":7553},[47,11134,2799],{"class":353},[47,11136,3151],{"class":118},[47,11138,3170],{"class":353},[47,11140,11141,11144,11147,11150,11152],{"class":49,"line":2856},[47,11142,11143],{"class":353},"    X[",[47,11145,11146],{"class":81},"~",[47,11148,11149],{"class":353},"mask] ",[47,11151,82],{"class":81},[47,11153,11154],{"class":353}," value\n",[47,11156,11157,11159],{"class":49,"line":2864},[47,11158,5057],{"class":81},[47,11160,10027],{"class":353},[47,11162,11163],{"class":49,"line":2875},[47,11164,326],{"emptyLinePlaceholder":325},[47,11166,11167,11169,11172,11174,11176,11178,11181],{"class":49,"line":2892},[47,11168,3003],{"class":81},[47,11170,11171],{"class":3006}," MaskedSoftmaxCELoss",[47,11173,1187],{"class":353},[47,11175,3013],{"class":3012},[47,11177,144],{"class":353},[47,11179,11180],{"class":3012},"CrossEntropyLoss",[47,11182,2805],{"class":353},[47,11184,11185],{"class":49,"line":2908},[47,11186,11187],{"class":2810},"    \"\"\"带遮蔽的softmax交叉熵损失函数\"\"\"\n",[47,11189,11190],{"class":49,"line":2916},[47,11191,11192],{"class":331},"    # pred的形状：(batch_size,num_steps,vocab_size)\n",[47,11194,11195],{"class":49,"line":2933},[47,11196,11197],{"class":331},"    # label的形状：(batch_size,num_steps)\n",[47,11199,11200],{"class":49,"line":2939},[47,11201,11202],{"class":331},"    # valid_len的形状：(batch_size,)\n",[47,11204,11205,11207,11209,11211,11213,11215,11218,11220,11223,11225,11227],{"class":49,"line":2964},[47,11206,3030],{"class":81},[47,11208,3123],{"class":53},[47,11210,1187],{"class":353},[47,11212,3039],{"class":3038},[47,11214,2799],{"class":353},[47,11216,11217],{"class":2795},"pred",[47,11219,2799],{"class":353},[47,11221,11222],{"class":2795},"label",[47,11224,2799],{"class":353},[47,11226,11061],{"class":2795},[47,11228,2805],{"class":353},[47,11230,11231,11234,11236],{"class":49,"line":2977},[47,11232,11233],{"class":353},"        weights ",[47,11235,82],{"class":81},[47,11237,11238],{"class":353}," torch.ones_like(label)\n",[47,11240,11241,11243,11245],{"class":49,"line":3215},[47,11242,11233],{"class":353},[47,11244,82],{"class":81},[47,11246,11247],{"class":353}," sequence_mask(weights, valid_len)\n",[47,11249,11250,11252,11255,11257,11259,11262],{"class":49,"line":5151},[47,11251,3084],{"class":3066},[47,11253,11254],{"class":353},".reduction",[47,11256,82],{"class":81},[47,11258,171],{"class":125},[47,11260,11261],{"class":57},"none",[47,11263,11264],{"class":125},"'\n",[47,11266,11267,11270,11272,11275,11278,11280],{"class":49,"line":5427},[47,11268,11269],{"class":353},"        unweighted_loss ",[47,11271,82],{"class":81},[47,11273,11274],{"class":3059}," super",[47,11276,11277],{"class":353},"(MaskedSoftmaxCELoss, ",[47,11279,3039],{"class":3066},[47,11281,11282],{"class":353},").forward(\n",[47,11284,11285,11288,11290,11292,11294,11296,11298],{"class":49,"line":5433},[47,11286,11287],{"class":353},"            pred.permute(",[47,11289,2141],{"class":118},[47,11291,2799],{"class":353},[47,11293,3192],{"class":118},[47,11295,2799],{"class":353},[47,11297,472],{"class":118},[47,11299,11300],{"class":353},"), label)\n",[47,11302,11303,11306,11308,11311,11313,11316,11318,11320,11322],{"class":49,"line":5439},[47,11304,11305],{"class":353},"        weighted_loss ",[47,11307,82],{"class":81},[47,11309,11310],{"class":353}," (unweighted_loss ",[47,11312,9099],{"class":81},[47,11314,11315],{"class":353}," weights).mean(",[47,11317,2845],{"class":2844},[47,11319,82],{"class":81},[47,11321,472],{"class":118},[47,11323,2853],{"class":353},[47,11325,11326,11328],{"class":49,"line":5445},[47,11327,2838],{"class":81},[47,11329,11330],{"class":353}," weighted_loss\n",[14,11332,11333],{},"梯度裁剪函数：",[37,11335,11337],{"className":2778,"code":11336,"language":2780,"meta":42,"style":42},"def grad_clipping(net, theta):\n    \"\"\"裁剪梯度\"\"\"\n    if isinstance(net, nn.Module):\n        params = [p for p in net.parameters() if p.requires_grad]\n    else:\n        params = net.params\n    norm = torch.sqrt(sum(torch.sum((p.grad ** 2)) for p in params))\n    if norm > theta:\n        for param in params:\n            param.grad[:] *= theta / norm\n",[44,11338,11339,11358,11363,11373,11399,11405,11414,11446,11459,11471],{"__ignoreMap":42},[47,11340,11341,11343,11346,11348,11351,11353,11356],{"class":49,"line":50},[47,11342,2787],{"class":81},[47,11344,11345],{"class":53}," grad_clipping",[47,11347,1187],{"class":353},[47,11349,11350],{"class":2795},"net",[47,11352,2799],{"class":353},[47,11354,11355],{"class":2795},"theta",[47,11357,2805],{"class":353},[47,11359,11360],{"class":49,"line":64},[47,11361,11362],{"class":2810},"    \"\"\"裁剪梯度\"\"\"\n",[47,11364,11365,11367,11370],{"class":49,"line":74},[47,11366,2821],{"class":81},[47,11368,11369],{"class":67}," isinstance",[47,11371,11372],{"class":353},"(net, nn.Module):\n",[47,11374,11375,11378,11380,11383,11385,11388,11390,11393,11396],{"class":49,"line":335},[47,11376,11377],{"class":353},"        params ",[47,11379,82],{"class":81},[47,11381,11382],{"class":353}," [p ",[47,11384,10897],{"class":81},[47,11386,11387],{"class":353}," p ",[47,11389,9828],{"class":81},[47,11391,11392],{"class":353}," net.parameters() ",[47,11394,11395],{"class":81},"if",[47,11397,11398],{"class":353}," p.requires_grad]\n",[47,11400,11401,11403],{"class":49,"line":344},[47,11402,2859],{"class":81},[47,11404,2833],{"class":353},[47,11406,11407,11409,11411],{"class":49,"line":2856},[47,11408,11377],{"class":353},[47,11410,82],{"class":81},[47,11412,11413],{"class":353}," net.params\n",[47,11415,11416,11419,11421,11424,11427,11430,11432,11435,11437,11439,11441,11443],{"class":49,"line":2864},[47,11417,11418],{"class":353},"    norm ",[47,11420,82],{"class":81},[47,11422,11423],{"class":353}," torch.sqrt(",[47,11425,11426],{"class":67},"sum",[47,11428,11429],{"class":353},"(torch.sum((p.grad ",[47,11431,3049],{"class":81},[47,11433,11434],{"class":118}," 2",[47,11436,3195],{"class":353},[47,11438,10897],{"class":81},[47,11440,11387],{"class":353},[47,11442,9828],{"class":81},[47,11444,11445],{"class":353}," params))\n",[47,11447,11448,11450,11453,11456],{"class":49,"line":2875},[47,11449,2821],{"class":81},[47,11451,11452],{"class":353}," norm ",[47,11454,11455],{"class":81},">",[47,11457,11458],{"class":353}," theta:\n",[47,11460,11461,11463,11466,11468],{"class":49,"line":2892},[47,11462,9822],{"class":81},[47,11464,11465],{"class":353}," param ",[47,11467,9828],{"class":81},[47,11469,11470],{"class":353}," params:\n",[47,11472,11473,11476,11478,11480,11483,11486,11488],{"class":49,"line":2908},[47,11474,11475],{"class":353},"            param.grad[",[47,11477,7554],{"class":7553},[47,11479,7570],{"class":353},[47,11481,11482],{"class":81},"*=",[47,11484,11485],{"class":353}," theta ",[47,11487,141],{"class":81},[47,11489,11490],{"class":353}," norm\n",[14,11492,11493,11494],{},"之后，在训练时，特定的序列开始词元（“",[11495,11496,11497,11498],"bos",{},"”）和 原始的输出序列（不包括序列结束词元“",[11499,11500,11501],"eos",{},"”） 拼接在一起作为解码器的输入。 这被称为强制教学（teacher forcing）， 因为原始的输出序列（词元的标签）被送入解码器。 或者，将来自上一个时间步的预测得到的词元作为解码器的当前输入。",[37,11503,11505],{"className":2778,"code":11504,"language":2780,"meta":42,"style":42},"def train_seq2seq(net, data_iter, lr, num_epochs, tgt_vocab, device):\n    \"\"\"训练序列到序列模型\"\"\"\n    def xavier_init_weights(m):\n        if type(m) == nn.Linear:\n            nn.init.xavier_uniform_(m.weight)\n        if type(m) == nn.GRU:\n            for param in m._flat_weights_names:\n                if \"weight\" in param:\n                    nn.init.xavier_uniform_(m._parameters[param])\n\n    net.apply(xavier_init_weights)\n    net.to(device)\n    optimizer = torch.optim.Adam(net.parameters(), lr=lr)\n    loss = MaskedSoftmaxCELoss()\n    net.train()\n    animator = d2l.Animator(xlabel='epoch', ylabel='loss',\n                     xlim=[10, num_epochs])\n    for epoch in range(num_epochs):\n        timer = d2l.Timer()\n        metric = d2l.Accumulator(2)  # 训练损失总和，词元数量\n        for batch in data_iter:\n            optimizer.zero_grad()\n            X, X_valid_len, Y, Y_valid_len = [x.to(device) for x in batch]\n            bos = torch.tensor([tgt_vocab['\u003Cbos>']] * Y.shape[0],\n                          device=device).reshape(-1, 1)\n            dec_input = torch.cat([bos, Y[:, :-1]], 1)  # 强制教学\n            Y_hat, _ = net(X, dec_input, X_valid_len)\n            l = loss(Y_hat, Y, Y_valid_len)\n            l.sum().backward()      # 损失函数的标量进行“反向传播”\n            grad_clipping(net, 1)\n            num_tokens = Y_valid_len.sum()\n            optimizer.step()\n            with torch.no_grad():\n                metric.add(l.sum(), num_tokens)\n        if (epoch + 1) % 10 == 0:\n            animator.add(epoch + 1, (metric[0] / metric[1],))\n    print(f'loss {metric[0] / metric[1]:.3f}, {metric[1] / timer.stop():.1f} '\n        f'tokens/sec on {str(device)}')\n",[44,11506,11507,11544,11549,11562,11577,11582,11600,11612,11630,11635,11639,11644,11649,11666,11676,11681,11719,11735,11750,11760,11778,11790,11795,11815,11845,11865,11895,11905,11915,11923,11932,11942,11947,11955,11960,11987,12013,12074],{"__ignoreMap":42},[47,11508,11509,11511,11514,11516,11518,11520,11523,11525,11528,11530,11533,11535,11538,11540,11542],{"class":49,"line":50},[47,11510,2787],{"class":81},[47,11512,11513],{"class":53}," train_seq2seq",[47,11515,1187],{"class":353},[47,11517,11350],{"class":2795},[47,11519,2799],{"class":353},[47,11521,11522],{"class":2795},"data_iter",[47,11524,2799],{"class":353},[47,11526,11527],{"class":2795},"lr",[47,11529,2799],{"class":353},[47,11531,11532],{"class":2795},"num_epochs",[47,11534,2799],{"class":353},[47,11536,11537],{"class":2795},"tgt_vocab",[47,11539,2799],{"class":353},[47,11541,10432],{"class":2795},[47,11543,2805],{"class":353},[47,11545,11546],{"class":49,"line":64},[47,11547,11548],{"class":2810},"    \"\"\"训练序列到序列模型\"\"\"\n",[47,11550,11551,11553,11556,11558,11560],{"class":49,"line":74},[47,11552,3030],{"class":81},[47,11554,11555],{"class":53}," xavier_init_weights",[47,11557,1187],{"class":353},[47,11559,755],{"class":2795},[47,11561,2805],{"class":353},[47,11563,11564,11566,11569,11572,11574],{"class":49,"line":335},[47,11565,2878],{"class":81},[47,11567,11568],{"class":3059}," type",[47,11570,11571],{"class":353},"(m) ",[47,11573,2884],{"class":81},[47,11575,11576],{"class":353}," nn.Linear:\n",[47,11578,11579],{"class":49,"line":344},[47,11580,11581],{"class":353},"            nn.init.xavier_uniform_(m.weight)\n",[47,11583,11584,11586,11588,11590,11592,11595,11598],{"class":49,"line":2856},[47,11585,2878],{"class":81},[47,11587,11568],{"class":3059},[47,11589,11571],{"class":353},[47,11591,2884],{"class":81},[47,11593,11594],{"class":353}," nn.",[47,11596,11597],{"class":118},"GRU",[47,11599,2833],{"class":353},[47,11601,11602,11605,11607,11609],{"class":49,"line":2864},[47,11603,11604],{"class":81},"            for",[47,11606,11465],{"class":353},[47,11608,9828],{"class":81},[47,11610,11611],{"class":353}," m._flat_weights_names:\n",[47,11613,11614,11617,11619,11622,11624,11627],{"class":49,"line":2875},[47,11615,11616],{"class":81},"                if",[47,11618,126],{"class":125},[47,11620,11621],{"class":57},"weight",[47,11623,9845],{"class":125},[47,11625,11626],{"class":81}," in",[47,11628,11629],{"class":353}," param:\n",[47,11631,11632],{"class":49,"line":2892},[47,11633,11634],{"class":353},"                    nn.init.xavier_uniform_(m._parameters[param])\n",[47,11636,11637],{"class":49,"line":2908},[47,11638,326],{"emptyLinePlaceholder":325},[47,11640,11641],{"class":49,"line":2916},[47,11642,11643],{"class":353},"    net.apply(xavier_init_weights)\n",[47,11645,11646],{"class":49,"line":2933},[47,11647,11648],{"class":353},"    net.to(device)\n",[47,11650,11651,11654,11656,11659,11661,11663],{"class":49,"line":2939},[47,11652,11653],{"class":353},"    optimizer ",[47,11655,82],{"class":81},[47,11657,11658],{"class":353}," torch.optim.Adam(net.parameters(), ",[47,11660,11527],{"class":2844},[47,11662,82],{"class":81},[47,11664,11665],{"class":353},"lr)\n",[47,11667,11668,11671,11673],{"class":49,"line":2964},[47,11669,11670],{"class":353},"    loss ",[47,11672,82],{"class":81},[47,11674,11675],{"class":353}," MaskedSoftmaxCELoss()\n",[47,11677,11678],{"class":49,"line":2977},[47,11679,11680],{"class":353},"    net.train()\n",[47,11682,11683,11686,11688,11691,11694,11696,11698,11701,11703,11705,11708,11710,11712,11715,11717],{"class":49,"line":3215},[47,11684,11685],{"class":353},"    animator ",[47,11687,82],{"class":81},[47,11689,11690],{"class":353}," d2l.Animator(",[47,11692,11693],{"class":2844},"xlabel",[47,11695,82],{"class":81},[47,11697,171],{"class":125},[47,11699,11700],{"class":57},"epoch",[47,11702,171],{"class":125},[47,11704,2799],{"class":353},[47,11706,11707],{"class":2844},"ylabel",[47,11709,82],{"class":81},[47,11711,171],{"class":125},[47,11713,11714],{"class":57},"loss",[47,11716,171],{"class":125},[47,11718,4196],{"class":353},[47,11720,11721,11724,11726,11729,11732],{"class":49,"line":5151},[47,11722,11723],{"class":2844},"                     xlim",[47,11725,82],{"class":81},[47,11727,11728],{"class":353},"[",[47,11730,11731],{"class":118},"10",[47,11733,11734],{"class":353},", num_epochs])\n",[47,11736,11737,11740,11743,11745,11747],{"class":49,"line":5427},[47,11738,11739],{"class":81},"    for",[47,11741,11742],{"class":353}," epoch ",[47,11744,9828],{"class":81},[47,11746,9831],{"class":67},[47,11748,11749],{"class":353},"(num_epochs):\n",[47,11751,11752,11755,11757],{"class":49,"line":5433},[47,11753,11754],{"class":353},"        timer ",[47,11756,82],{"class":81},[47,11758,11759],{"class":353}," d2l.Timer()\n",[47,11761,11762,11765,11767,11770,11772,11775],{"class":49,"line":5439},[47,11763,11764],{"class":353},"        metric ",[47,11766,82],{"class":81},[47,11768,11769],{"class":353}," d2l.Accumulator(",[47,11771,3192],{"class":118},[47,11773,11774],{"class":353},")  ",[47,11776,11777],{"class":331},"# 训练损失总和，词元数量\n",[47,11779,11780,11782,11785,11787],{"class":49,"line":5445},[47,11781,9822],{"class":81},[47,11783,11784],{"class":353}," batch ",[47,11786,9828],{"class":81},[47,11788,11789],{"class":353}," data_iter:\n",[47,11791,11792],{"class":49,"line":5466},[47,11793,11794],{"class":353},"            optimizer.zero_grad()\n",[47,11796,11797,11800,11802,11805,11807,11810,11812],{"class":49,"line":5485},[47,11798,11799],{"class":353},"            X, X_valid_len, Y, Y_valid_len ",[47,11801,82],{"class":81},[47,11803,11804],{"class":353}," [x.to(device) ",[47,11806,10897],{"class":81},[47,11808,11809],{"class":353}," x ",[47,11811,9828],{"class":81},[47,11813,11814],{"class":353}," batch]\n",[47,11816,11817,11820,11822,11825,11827,11830,11832,11835,11837,11840,11842],{"class":49,"line":5504},[47,11818,11819],{"class":353},"            bos ",[47,11821,82],{"class":81},[47,11823,11824],{"class":353}," torch.tensor([tgt_vocab[",[47,11826,171],{"class":125},[47,11828,11829],{"class":57},"\u003Cbos>",[47,11831,171],{"class":125},[47,11833,11834],{"class":353},"]] ",[47,11836,9099],{"class":81},[47,11838,11839],{"class":353}," Y.shape[",[47,11841,2141],{"class":118},[47,11843,11844],{"class":353},"],\n",[47,11846,11847,11850,11852,11855,11857,11859,11861,11863],{"class":49,"line":5509},[47,11848,11849],{"class":2844},"                          device",[47,11851,82],{"class":81},[47,11853,11854],{"class":353},"device).reshape(",[47,11856,2926],{"class":81},[47,11858,472],{"class":118},[47,11860,2799],{"class":353},[47,11862,472],{"class":118},[47,11864,2853],{"class":353},[47,11866,11867,11870,11872,11875,11877,11879,11881,11883,11885,11888,11890,11892],{"class":49,"line":5525},[47,11868,11869],{"class":353},"            dec_input ",[47,11871,82],{"class":81},[47,11873,11874],{"class":353}," torch.cat([bos, Y[",[47,11876,7554],{"class":7553},[47,11878,2799],{"class":353},[47,11880,7554],{"class":7553},[47,11882,2926],{"class":81},[47,11884,472],{"class":118},[47,11886,11887],{"class":353},"]], ",[47,11889,472],{"class":118},[47,11891,11774],{"class":353},[47,11893,11894],{"class":331},"# 强制教学\n",[47,11896,11897,11900,11902],{"class":49,"line":5531},[47,11898,11899],{"class":353},"            Y_hat, _ ",[47,11901,82],{"class":81},[47,11903,11904],{"class":353}," net(X, dec_input, X_valid_len)\n",[47,11906,11907,11910,11912],{"class":49,"line":5537},[47,11908,11909],{"class":353},"            l ",[47,11911,82],{"class":81},[47,11913,11914],{"class":353}," loss(Y_hat, Y, Y_valid_len)\n",[47,11916,11917,11920],{"class":49,"line":5547},[47,11918,11919],{"class":353},"            l.sum().backward()      ",[47,11921,11922],{"class":331},"# 损失函数的标量进行“反向传播”\n",[47,11924,11925,11928,11930],{"class":49,"line":5571},[47,11926,11927],{"class":353},"            grad_clipping(net, ",[47,11929,472],{"class":118},[47,11931,2853],{"class":353},[47,11933,11934,11937,11939],{"class":49,"line":5576},[47,11935,11936],{"class":353},"            num_tokens ",[47,11938,82],{"class":81},[47,11940,11941],{"class":353}," Y_valid_len.sum()\n",[47,11943,11944],{"class":49,"line":5582},[47,11945,11946],{"class":353},"            optimizer.step()\n",[47,11948,11949,11952],{"class":49,"line":5596},[47,11950,11951],{"class":81},"            with",[47,11953,11954],{"class":353}," torch.no_grad():\n",[47,11956,11957],{"class":49,"line":5601},[47,11958,11959],{"class":353},"                metric.add(l.sum(), num_tokens)\n",[47,11961,11962,11964,11967,11969,11971,11973,11976,11979,11982,11985],{"class":49,"line":5607},[47,11963,2878],{"class":81},[47,11965,11966],{"class":353}," (epoch ",[47,11968,6147],{"class":81},[47,11970,2887],{"class":118},[47,11972,7516],{"class":353},[47,11974,11975],{"class":81},"%",[47,11977,11978],{"class":118}," 10",[47,11980,11981],{"class":81}," ==",[47,11983,11984],{"class":118}," 0",[47,11986,2833],{"class":353},[47,11988,11989,11992,11994,11996,11999,12001,12003,12005,12008,12010],{"class":49,"line":5622},[47,11990,11991],{"class":353},"            animator.add(epoch ",[47,11993,6147],{"class":81},[47,11995,2887],{"class":118},[47,11997,11998],{"class":353},", (metric[",[47,12000,2141],{"class":118},[47,12002,7570],{"class":353},[47,12004,141],{"class":81},[47,12006,12007],{"class":353}," metric[",[47,12009,472],{"class":118},[47,12011,12012],{"class":353},"],))\n",[47,12014,12015,12018,12020,12023,12026,12029,12032,12034,12036,12038,12040,12042,12045,12048,12051,12053,12055,12057,12059,12061,12063,12066,12069,12071],{"class":49,"line":9911},[47,12016,12017],{"class":67},"    print",[47,12019,1187],{"class":353},[47,12021,12022],{"class":81},"f",[47,12024,12025],{"class":57},"'loss ",[47,12027,12028],{"class":118},"{",[47,12030,12031],{"class":353},"metric[",[47,12033,2141],{"class":118},[47,12035,7570],{"class":353},[47,12037,141],{"class":81},[47,12039,12007],{"class":353},[47,12041,472],{"class":118},[47,12043,12044],{"class":353},"]",[47,12046,12047],{"class":81},":.3f",[47,12049,12050],{"class":118},"}",[47,12052,2799],{"class":57},[47,12054,12028],{"class":118},[47,12056,12031],{"class":353},[47,12058,472],{"class":118},[47,12060,7570],{"class":353},[47,12062,141],{"class":81},[47,12064,12065],{"class":353}," timer.stop()",[47,12067,12068],{"class":81},":.1f",[47,12070,12050],{"class":118},[47,12072,12073],{"class":57}," '\n",[47,12075,12076,12079,12082,12084,12086,12089,12091,12093],{"class":49,"line":9917},[47,12077,12078],{"class":81},"        f",[47,12080,12081],{"class":57},"'tokens/sec on ",[47,12083,12028],{"class":118},[47,12085,9855],{"class":3059},[47,12087,12088],{"class":353},"(device)",[47,12090,12050],{"class":118},[47,12092,171],{"class":57},[47,12094,2853],{"class":353},[14,12096,12097],{},"进行数据集的数据处理",[37,12099,12101],{"className":2778,"code":12100,"language":2780,"meta":42,"style":42},"def read_data_nmt():\n    \"\"\"载入“英语－法语”数据集\n\n    Defined in :numref:`sec_machine_translation`\"\"\"\n    data_dir = d2l.download_extract('fra-eng')\n    with open(os.path.join(data_dir, 'fra.txt'), 'r',\n             encoding='utf-8') as f:\n        return f.read()\n\ndef preprocess_nmt(text):\n    \"\"\"预处理“英语－法语”数据集\n\n    Defined in :numref:`sec_machine_translation`\"\"\"\n    def no_space(char, prev_char):\n        return char in set(',.!?') and prev_char != ' '\n\n    # 使用空格替换不间断空格\n    # 使用小写字母替换大写字母\n    text = text.replace('\\u202f', ' ').replace('\\xa0', ' ').lower()\n    # 在单词和标点符号之间插入空格\n    out = [' ' + char if i > 0 and no_space(char, text[i - 1]) else char\n           for i, char in enumerate(text)]\n    return ''.join(out)\n\ndef tokenize_nmt(text, num_examples=None):\n    \"\"\"词元化“英语－法语”数据数据集\n\n    Defined in :numref:`sec_machine_translation`\"\"\"\n    source, target = [], []\n    for i, line in enumerate(text.split('\\n')):\n        if num_examples and i > num_examples:\n            break\n        parts = line.split('\\t')\n        if len(parts) == 2:\n            source.append(parts[0].split(' '))\n            target.append(parts[1].split(' '))\n    return source, target\n\ndef load_data_nmt(batch_size, num_steps, num_examples=600):\n    \"\"\"返回翻译数据集的迭代器和词表\n\n    Defined in :numref:`subsec_mt_data_loading`\"\"\"\n    text = preprocess_nmt(read_data_nmt())\n    source, target = tokenize_nmt(text, num_examples)\n    src_vocab = d2l.Vocab(source, min_freq=2,\n                          reserved_tokens=['\u003Cpad>', '\u003Cbos>', '\u003Ceos>'])\n    tgt_vocab = d2l.Vocab(target, min_freq=2,\n                          reserved_tokens=['\u003Cpad>', '\u003Cbos>', '\u003Ceos>'])\n    src_array, src_valid_len = build_array_nmt(source, src_vocab, num_steps)\n    tgt_array, tgt_valid_len = build_array_nmt(target, tgt_vocab, num_steps)\n    data_arrays = (src_array, src_valid_len, tgt_array, tgt_valid_len)\n    data_iter = d2l.load_array(data_arrays, batch_size)\n    return data_iter, src_vocab, tgt_vocab\n",[44,12102,12103,12113,12118,12122,12127,12146,12175,12197,12204,12208,12222,12227,12231,12235,12254,12290,12294,12299,12304,12347,12352,12397,12412,12422,12426,12448,12453,12457,12461,12471,12495,12511,12516,12535,12550,12567,12582,12589,12593,12621,12626,12630,12635,12644,12653,12672,12707,12725,12757,12767,12777,12787,12797],{"__ignoreMap":42},[47,12104,12105,12107,12110],{"class":49,"line":50},[47,12106,2787],{"class":81},[47,12108,12109],{"class":53}," read_data_nmt",[47,12111,12112],{"class":353},"():\n",[47,12114,12115],{"class":49,"line":64},[47,12116,12117],{"class":2810},"    \"\"\"载入“英语－法语”数据集\n",[47,12119,12120],{"class":49,"line":74},[47,12121,326],{"emptyLinePlaceholder":325},[47,12123,12124],{"class":49,"line":335},[47,12125,12126],{"class":2810},"    Defined in :numref:`sec_machine_translation`\"\"\"\n",[47,12128,12129,12132,12134,12137,12139,12142,12144],{"class":49,"line":344},[47,12130,12131],{"class":353},"    data_dir ",[47,12133,82],{"class":81},[47,12135,12136],{"class":353}," d2l.download_extract(",[47,12138,171],{"class":125},[47,12140,12141],{"class":57},"fra-eng",[47,12143,171],{"class":125},[47,12145,2853],{"class":353},[47,12147,12148,12151,12154,12157,12159,12162,12164,12167,12169,12171,12173],{"class":49,"line":2856},[47,12149,12150],{"class":81},"    with",[47,12152,12153],{"class":67}," open",[47,12155,12156],{"class":353},"(os.path.join(data_dir, ",[47,12158,171],{"class":125},[47,12160,12161],{"class":57},"fra.txt",[47,12163,171],{"class":125},[47,12165,12166],{"class":353},"), ",[47,12168,171],{"class":125},[47,12170,8196],{"class":57},[47,12172,171],{"class":125},[47,12174,4196],{"class":353},[47,12176,12177,12180,12182,12184,12187,12189,12191,12194],{"class":49,"line":2864},[47,12178,12179],{"class":2844},"             encoding",[47,12181,82],{"class":81},[47,12183,171],{"class":125},[47,12185,12186],{"class":57},"utf-8",[47,12188,171],{"class":125},[47,12190,7516],{"class":353},[47,12192,12193],{"class":81},"as",[47,12195,12196],{"class":353}," f:\n",[47,12198,12199,12201],{"class":49,"line":2875},[47,12200,2838],{"class":81},[47,12202,12203],{"class":353}," f.read()\n",[47,12205,12206],{"class":49,"line":2892},[47,12207,326],{"emptyLinePlaceholder":325},[47,12209,12210,12212,12215,12217,12220],{"class":49,"line":2908},[47,12211,2787],{"class":81},[47,12213,12214],{"class":53}," preprocess_nmt",[47,12216,1187],{"class":353},[47,12218,12219],{"class":2795},"text",[47,12221,2805],{"class":353},[47,12223,12224],{"class":49,"line":2916},[47,12225,12226],{"class":2810},"    \"\"\"预处理“英语－法语”数据集\n",[47,12228,12229],{"class":49,"line":2933},[47,12230,326],{"emptyLinePlaceholder":325},[47,12232,12233],{"class":49,"line":2939},[47,12234,12126],{"class":2810},[47,12236,12237,12239,12242,12244,12247,12249,12252],{"class":49,"line":2964},[47,12238,3030],{"class":81},[47,12240,12241],{"class":53}," no_space",[47,12243,1187],{"class":353},[47,12245,12246],{"class":2795},"char",[47,12248,2799],{"class":353},[47,12250,12251],{"class":2795},"prev_char",[47,12253,2805],{"class":353},[47,12255,12256,12258,12261,12263,12266,12268,12270,12273,12275,12277,12280,12283,12286,12288],{"class":49,"line":2977},[47,12257,2838],{"class":81},[47,12259,12260],{"class":353}," char ",[47,12262,9828],{"class":81},[47,12264,12265],{"class":3059}," set",[47,12267,1187],{"class":353},[47,12269,171],{"class":125},[47,12271,12272],{"class":57},",.!?",[47,12274,171],{"class":125},[47,12276,7516],{"class":353},[47,12278,12279],{"class":81},"and",[47,12281,12282],{"class":353}," prev_char ",[47,12284,12285],{"class":81},"!=",[47,12287,165],{"class":125},[47,12289,12073],{"class":125},[47,12291,12292],{"class":49,"line":3215},[47,12293,326],{"emptyLinePlaceholder":325},[47,12295,12296],{"class":49,"line":5151},[47,12297,12298],{"class":331},"    # 使用空格替换不间断空格\n",[47,12300,12301],{"class":49,"line":5427},[47,12302,12303],{"class":331},"    # 使用小写字母替换大写字母\n",[47,12305,12306,12309,12311,12314,12316,12320,12322,12324,12326,12328,12331,12333,12336,12338,12340,12342,12344],{"class":49,"line":5433},[47,12307,12308],{"class":353},"    text ",[47,12310,82],{"class":81},[47,12312,12313],{"class":353}," text.replace(",[47,12315,171],{"class":125},[47,12317,12319],{"class":12318},"sRfyP","\\u202f",[47,12321,171],{"class":125},[47,12323,2799],{"class":353},[47,12325,171],{"class":125},[47,12327,165],{"class":125},[47,12329,12330],{"class":353},").replace(",[47,12332,171],{"class":125},[47,12334,12335],{"class":12318},"\\xa0",[47,12337,171],{"class":125},[47,12339,2799],{"class":353},[47,12341,171],{"class":125},[47,12343,165],{"class":125},[47,12345,12346],{"class":353},").lower()\n",[47,12348,12349],{"class":49,"line":5439},[47,12350,12351],{"class":331},"    # 在单词和标点符号之间插入空格\n",[47,12353,12354,12357,12359,12361,12363,12365,12368,12370,12372,12374,12376,12378,12381,12384,12386,12388,12391,12394],{"class":49,"line":5445},[47,12355,12356],{"class":353},"    out ",[47,12358,82],{"class":81},[47,12360,9959],{"class":353},[47,12362,171],{"class":125},[47,12364,165],{"class":125},[47,12366,12367],{"class":81}," +",[47,12369,12260],{"class":353},[47,12371,11395],{"class":81},[47,12373,9825],{"class":353},[47,12375,11455],{"class":81},[47,12377,11984],{"class":118},[47,12379,12380],{"class":81}," and",[47,12382,12383],{"class":353}," no_space(char, text[i ",[47,12385,2926],{"class":81},[47,12387,2887],{"class":118},[47,12389,12390],{"class":353},"]) ",[47,12392,12393],{"class":81},"else",[47,12395,12396],{"class":353}," char\n",[47,12398,12399,12402,12405,12407,12409],{"class":49,"line":5466},[47,12400,12401],{"class":81},"           for",[47,12403,12404],{"class":353}," i, char ",[47,12406,9828],{"class":81},[47,12408,9988],{"class":67},[47,12410,12411],{"class":353},"(text)]\n",[47,12413,12414,12416,12419],{"class":49,"line":5485},[47,12415,5057],{"class":81},[47,12417,12418],{"class":125}," ''",[47,12420,12421],{"class":353},".join(out)\n",[47,12423,12424],{"class":49,"line":5504},[47,12425,326],{"emptyLinePlaceholder":325},[47,12427,12428,12430,12433,12435,12437,12439,12442,12444,12446],{"class":49,"line":5509},[47,12429,2787],{"class":81},[47,12431,12432],{"class":53}," tokenize_nmt",[47,12434,1187],{"class":353},[47,12436,12219],{"class":2795},[47,12438,2799],{"class":353},[47,12440,12441],{"class":2795},"num_examples",[47,12443,82],{"class":81},[47,12445,3151],{"class":118},[47,12447,2805],{"class":353},[47,12449,12450],{"class":49,"line":5525},[47,12451,12452],{"class":2810},"    \"\"\"词元化“英语－法语”数据数据集\n",[47,12454,12455],{"class":49,"line":5531},[47,12456,326],{"emptyLinePlaceholder":325},[47,12458,12459],{"class":49,"line":5537},[47,12460,12126],{"class":2810},[47,12462,12463,12466,12468],{"class":49,"line":5547},[47,12464,12465],{"class":353},"    source, target ",[47,12467,82],{"class":81},[47,12469,12470],{"class":353}," [], []\n",[47,12472,12473,12475,12478,12480,12482,12485,12487,12490,12492],{"class":49,"line":5571},[47,12474,11739],{"class":81},[47,12476,12477],{"class":353}," i, line ",[47,12479,9828],{"class":81},[47,12481,9988],{"class":67},[47,12483,12484],{"class":353},"(text.split(",[47,12486,171],{"class":125},[47,12488,12489],{"class":12318},"\\n",[47,12491,171],{"class":125},[47,12493,12494],{"class":353},")):\n",[47,12496,12497,12499,12502,12504,12506,12508],{"class":49,"line":5576},[47,12498,2878],{"class":81},[47,12500,12501],{"class":353}," num_examples ",[47,12503,12279],{"class":81},[47,12505,9825],{"class":353},[47,12507,11455],{"class":81},[47,12509,12510],{"class":353}," num_examples:\n",[47,12512,12513],{"class":49,"line":5582},[47,12514,12515],{"class":81},"            break\n",[47,12517,12518,12521,12523,12526,12528,12531,12533],{"class":49,"line":5596},[47,12519,12520],{"class":353},"        parts ",[47,12522,82],{"class":81},[47,12524,12525],{"class":353}," line.split(",[47,12527,171],{"class":125},[47,12529,12530],{"class":12318},"\\t",[47,12532,171],{"class":125},[47,12534,2853],{"class":353},[47,12536,12537,12539,12541,12544,12546,12548],{"class":49,"line":5601},[47,12538,2878],{"class":81},[47,12540,9968],{"class":67},[47,12542,12543],{"class":353},"(parts) ",[47,12545,2884],{"class":81},[47,12547,11434],{"class":118},[47,12549,2833],{"class":353},[47,12551,12552,12555,12557,12560,12562,12564],{"class":49,"line":5607},[47,12553,12554],{"class":353},"            source.append(parts[",[47,12556,2141],{"class":118},[47,12558,12559],{"class":353},"].split(",[47,12561,171],{"class":125},[47,12563,165],{"class":125},[47,12565,12566],{"class":353},"))\n",[47,12568,12569,12572,12574,12576,12578,12580],{"class":49,"line":5622},[47,12570,12571],{"class":353},"            target.append(parts[",[47,12573,472],{"class":118},[47,12575,12559],{"class":353},[47,12577,171],{"class":125},[47,12579,165],{"class":125},[47,12581,12566],{"class":353},[47,12583,12584,12586],{"class":49,"line":9911},[47,12585,5057],{"class":81},[47,12587,12588],{"class":353}," source, target\n",[47,12590,12591],{"class":49,"line":9917},[47,12592,326],{"emptyLinePlaceholder":325},[47,12594,12595,12597,12600,12602,12605,12607,12610,12612,12614,12616,12619],{"class":49,"line":9923},[47,12596,2787],{"class":81},[47,12598,12599],{"class":53}," load_data_nmt",[47,12601,1187],{"class":353},[47,12603,12604],{"class":2795},"batch_size",[47,12606,2799],{"class":353},[47,12608,12609],{"class":2795},"num_steps",[47,12611,2799],{"class":353},[47,12613,12441],{"class":2795},[47,12615,82],{"class":81},[47,12617,12618],{"class":118},"600",[47,12620,2805],{"class":353},[47,12622,12623],{"class":49,"line":9950},[47,12624,12625],{"class":2810},"    \"\"\"返回翻译数据集的迭代器和词表\n",[47,12627,12628],{"class":49,"line":9978},[47,12629,326],{"emptyLinePlaceholder":325},[47,12631,12632],{"class":49,"line":9998},[47,12633,12634],{"class":2810},"    Defined in :numref:`subsec_mt_data_loading`\"\"\"\n",[47,12636,12637,12639,12641],{"class":49,"line":10009},[47,12638,12308],{"class":353},[47,12640,82],{"class":81},[47,12642,12643],{"class":353}," preprocess_nmt(read_data_nmt())\n",[47,12645,12646,12648,12650],{"class":49,"line":10022},[47,12647,12465],{"class":353},[47,12649,82],{"class":81},[47,12651,12652],{"class":353}," tokenize_nmt(text, num_examples)\n",[47,12654,12655,12658,12660,12663,12666,12668,12670],{"class":49,"line":10525},[47,12656,12657],{"class":353},"    src_vocab ",[47,12659,82],{"class":81},[47,12661,12662],{"class":353}," d2l.Vocab(source, ",[47,12664,12665],{"class":2844},"min_freq",[47,12667,82],{"class":81},[47,12669,3192],{"class":118},[47,12671,4196],{"class":353},[47,12673,12674,12677,12679,12681,12683,12686,12688,12690,12692,12694,12696,12698,12700,12703,12705],{"class":49,"line":10540},[47,12675,12676],{"class":2844},"                          reserved_tokens",[47,12678,82],{"class":81},[47,12680,11728],{"class":353},[47,12682,171],{"class":125},[47,12684,12685],{"class":57},"\u003Cpad>",[47,12687,171],{"class":125},[47,12689,2799],{"class":353},[47,12691,171],{"class":125},[47,12693,11829],{"class":57},[47,12695,171],{"class":125},[47,12697,2799],{"class":353},[47,12699,171],{"class":125},[47,12701,12702],{"class":57},"\u003Ceos>",[47,12704,171],{"class":125},[47,12706,2905],{"class":353},[47,12708,12709,12712,12714,12717,12719,12721,12723],{"class":49,"line":10545},[47,12710,12711],{"class":353},"    tgt_vocab ",[47,12713,82],{"class":81},[47,12715,12716],{"class":353}," d2l.Vocab(target, ",[47,12718,12665],{"class":2844},[47,12720,82],{"class":81},[47,12722,3192],{"class":118},[47,12724,4196],{"class":353},[47,12726,12727,12729,12731,12733,12735,12737,12739,12741,12743,12745,12747,12749,12751,12753,12755],{"class":49,"line":10560},[47,12728,12676],{"class":2844},[47,12730,82],{"class":81},[47,12732,11728],{"class":353},[47,12734,171],{"class":125},[47,12736,12685],{"class":57},[47,12738,171],{"class":125},[47,12740,2799],{"class":353},[47,12742,171],{"class":125},[47,12744,11829],{"class":57},[47,12746,171],{"class":125},[47,12748,2799],{"class":353},[47,12750,171],{"class":125},[47,12752,12702],{"class":57},[47,12754,171],{"class":125},[47,12756,2905],{"class":353},[47,12758,12759,12762,12764],{"class":49,"line":10589},[47,12760,12761],{"class":353},"    src_array, src_valid_len ",[47,12763,82],{"class":81},[47,12765,12766],{"class":353}," build_array_nmt(source, src_vocab, num_steps)\n",[47,12768,12769,12772,12774],{"class":49,"line":10608},[47,12770,12771],{"class":353},"    tgt_array, tgt_valid_len ",[47,12773,82],{"class":81},[47,12775,12776],{"class":353}," build_array_nmt(target, tgt_vocab, num_steps)\n",[47,12778,12779,12782,12784],{"class":49,"line":10629},[47,12780,12781],{"class":353},"    data_arrays ",[47,12783,82],{"class":81},[47,12785,12786],{"class":353}," (src_array, src_valid_len, tgt_array, tgt_valid_len)\n",[47,12788,12789,12792,12794],{"class":49,"line":10649},[47,12790,12791],{"class":353},"    data_iter ",[47,12793,82],{"class":81},[47,12795,12796],{"class":353}," d2l.load_array(data_arrays, batch_size)\n",[47,12798,12799,12801],{"class":49,"line":10660},[47,12800,5057],{"class":81},[47,12802,12803],{"class":353}," data_iter, src_vocab, tgt_vocab\n",[14,12805,12806],{},"之后按照 transformer 架构创建编码器-解码器架构，",[37,12808,12810],{"className":2778,"code":12809,"language":2780,"meta":42,"style":42},"num_hiddens, num_layers, dropout, batch_size, num_steps = 32, 2, 0.1, 64, 10\nlr, num_epochs, device = 0.005, 200, torch.device('cuda:0') if torch.cuda.device_count() >= 1 else torch.device('cpu')\nffn_num_hiddens, num_heads = 64, 4\n\ntrain_iter, src_vocab, tgt_vocab = d2l.load_data_nmt(batch_size, num_steps)\n\nencoder = TransformerEncoder(\n    len(src_vocab), num_hiddens, ffn_num_hiddens, num_heads, num_layers,\n    dropout)\ndecoder = TransformerDecoder(\n    len(tgt_vocab), num_hiddens, ffn_num_hiddens, num_heads, num_layers,\n    dropout)\nnet = EncoderDecoder(encoder, decoder)\ntrain_seq2seq(net, train_iter, lr, num_epochs, tgt_vocab, device)\n",[44,12811,12812,12841,12893,12908,12912,12922,12926,12936,12944,12949,12959,12966,12970,12980],{"__ignoreMap":42},[47,12813,12814,12817,12819,12822,12824,12826,12828,12831,12833,12836,12838],{"class":49,"line":50},[47,12815,12816],{"class":353},"num_hiddens, num_layers, dropout, batch_size, num_steps ",[47,12818,82],{"class":81},[47,12820,12821],{"class":118}," 32",[47,12823,2799],{"class":353},[47,12825,3192],{"class":118},[47,12827,2799],{"class":353},[47,12829,12830],{"class":118},"0.1",[47,12832,2799],{"class":353},[47,12834,12835],{"class":118},"64",[47,12837,2799],{"class":353},[47,12839,12840],{"class":118},"10\n",[47,12842,12843,12846,12848,12851,12853,12856,12859,12861,12864,12866,12868,12870,12873,12876,12878,12881,12884,12886,12889,12891],{"class":49,"line":64},[47,12844,12845],{"class":353},"lr, num_epochs, device ",[47,12847,82],{"class":81},[47,12849,12850],{"class":118}," 0.005",[47,12852,2799],{"class":353},[47,12854,12855],{"class":118},"200",[47,12857,12858],{"class":353},", torch.device(",[47,12860,171],{"class":125},[47,12862,12863],{"class":57},"cuda:0",[47,12865,171],{"class":125},[47,12867,7516],{"class":353},[47,12869,11395],{"class":81},[47,12871,12872],{"class":353}," torch.cuda.device_count() ",[47,12874,12875],{"class":81},">=",[47,12877,2887],{"class":118},[47,12879,12880],{"class":81}," else",[47,12882,12883],{"class":353}," torch.device(",[47,12885,171],{"class":125},[47,12887,12888],{"class":57},"cpu",[47,12890,171],{"class":125},[47,12892,2853],{"class":353},[47,12894,12895,12898,12900,12903,12905],{"class":49,"line":74},[47,12896,12897],{"class":353},"ffn_num_hiddens, num_heads ",[47,12899,82],{"class":81},[47,12901,12902],{"class":118}," 64",[47,12904,2799],{"class":353},[47,12906,12907],{"class":118},"4\n",[47,12909,12910],{"class":49,"line":335},[47,12911,326],{"emptyLinePlaceholder":325},[47,12913,12914,12917,12919],{"class":49,"line":344},[47,12915,12916],{"class":353},"train_iter, src_vocab, tgt_vocab ",[47,12918,82],{"class":81},[47,12920,12921],{"class":353}," d2l.load_data_nmt(batch_size, num_steps)\n",[47,12923,12924],{"class":49,"line":2856},[47,12925,326],{"emptyLinePlaceholder":325},[47,12927,12928,12931,12933],{"class":49,"line":2864},[47,12929,12930],{"class":353},"encoder ",[47,12932,82],{"class":81},[47,12934,12935],{"class":353}," TransformerEncoder(\n",[47,12937,12938,12941],{"class":49,"line":2875},[47,12939,12940],{"class":67},"    len",[47,12942,12943],{"class":353},"(src_vocab), num_hiddens, ffn_num_hiddens, num_heads, num_layers,\n",[47,12945,12946],{"class":49,"line":2892},[47,12947,12948],{"class":353},"    dropout)\n",[47,12950,12951,12954,12956],{"class":49,"line":2908},[47,12952,12953],{"class":353},"decoder ",[47,12955,82],{"class":81},[47,12957,12958],{"class":353}," TransformerDecoder(\n",[47,12960,12961,12963],{"class":49,"line":2916},[47,12962,12940],{"class":67},[47,12964,12965],{"class":353},"(tgt_vocab), num_hiddens, ffn_num_hiddens, num_heads, num_layers,\n",[47,12967,12968],{"class":49,"line":2933},[47,12969,12948],{"class":353},[47,12971,12972,12975,12977],{"class":49,"line":2939},[47,12973,12974],{"class":353},"net ",[47,12976,82],{"class":81},[47,12978,12979],{"class":353}," EncoderDecoder(encoder, decoder)\n",[47,12981,12982],{"class":49,"line":2964},[47,12983,12984],{"class":353},"train_seq2seq(net, train_iter, lr, num_epochs, tgt_vocab, device)\n",[10,12986,12987],{"id":12987},"参考文献",[14,12989,12990],{},[285,12991,424],{"href":422,"rel":12992},[289],[14,12994,12995],{},[285,12996,418],{"href":416,"rel":12997},[289],[368,12999,13000],{},"html pre.shiki code .szJfE, html code.shiki .szJfE{--shiki-default:#D73A49;--shiki-dark:#FF79C6}html pre.shiki code .sCLZk, html code.shiki .sCLZk{--shiki-default:#6F42C1;--shiki-dark:#50FA7B}html pre.shiki code .scbbO, html code.shiki .scbbO{--shiki-default:#24292E;--shiki-dark:#F8F8F2}html pre.shiki code .syNf4, html code.shiki .syNf4{--shiki-default:#24292E;--shiki-default-font-style:inherit;--shiki-dark:#FFB86C;--shiki-dark-font-style:italic}html pre.shiki code .seLWX, html code.shiki .seLWX{--shiki-default:#032F62;--shiki-dark:#6272A4}html pre.shiki code .sfgPZ, html code.shiki .sfgPZ{--shiki-default:#6A737D;--shiki-dark:#6272A4}html pre.shiki code .soDru, html code.shiki .soDru{--shiki-default:#005CC5;--shiki-dark:#BD93F9}html pre.shiki code .sQkXh, html code.shiki .sQkXh{--shiki-default:#E36209;--shiki-default-font-style:inherit;--shiki-dark:#FFB86C;--shiki-dark-font-style:italic}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html pre.shiki code .skCyd, html code.shiki .skCyd{--shiki-default:#6F42C1;--shiki-dark:#8BE9FD}html pre.shiki code .sDP9b, html code.shiki .sDP9b{--shiki-default:#6F42C1;--shiki-default-font-style:inherit;--shiki-dark:#8BE9FD;--shiki-dark-font-style:italic}html pre.shiki code .sD3jg, html code.shiki .sD3jg{--shiki-default:#24292E;--shiki-default-font-style:inherit;--shiki-dark:#BD93F9;--shiki-dark-font-style:italic}html pre.shiki code .sPGBF, html code.shiki .sPGBF{--shiki-default:#005CC5;--shiki-default-font-style:inherit;--shiki-dark:#8BE9FD;--shiki-dark-font-style:italic}html pre.shiki code .sJti5, html code.shiki .sJti5{--shiki-default:#005CC5;--shiki-default-font-style:inherit;--shiki-dark:#BD93F9;--shiki-dark-font-style:italic}html pre.shiki code .sDoOe, html code.shiki .sDoOe{--shiki-default:#24292E;--shiki-dark:#FF79C6}html pre.shiki code .sDgm9, html code.shiki .sDgm9{--shiki-default:#005CC5;--shiki-dark:#8BE9FD}html pre.shiki code .sMWOi, html code.shiki .sMWOi{--shiki-default:#032F62;--shiki-dark:#E9F284}html pre.shiki code .sEzAm, html code.shiki .sEzAm{--shiki-default:#032F62;--shiki-dark:#F1FA8C}html pre.shiki code .sRfyP, html code.shiki .sRfyP{--shiki-default:#005CC5;--shiki-dark:#FF79C6}",{"title":42,"searchDepth":64,"depth":64,"links":13002},[13003,13004,13005,13009,13010,13011,13012,13013,13014,13015,13016],{"id":12,"depth":64,"text":12},{"id":428,"depth":64,"text":428},{"id":438,"depth":64,"text":438,"children":13006},[13007,13008],{"id":441,"depth":74,"text":441},{"id":3240,"depth":74,"text":3240},{"id":5632,"depth":64,"text":5632},{"id":7667,"depth":64,"text":7667},{"id":8163,"depth":64,"text":8163},{"id":8500,"depth":64,"text":8500},{"id":9414,"depth":64,"text":9415},{"id":10030,"depth":64,"text":10031},{"id":11037,"depth":64,"text":11037},{"id":12987,"depth":64,"text":12987},"距离首次学习 transformer 已经过去一年，内容忘的差不多了，决定复习一下。",{"date":13019,"image":13020,"alt":403,"tags":13021,"published":325},"30st Jan 2026","/blogs-img/blog1.jpg",[13022,13023],"deep-learning","学习笔记","/blogs/transformer",{"title":403,"description":13017},"blogs/1. transformer复习笔记","59tIbWfqdpJGJRh5PHVoEHC170F6vnech8kVXdG8MGw",{"id":13029,"title":13030,"body":13031,"description":44301,"extension":388,"meta":44302,"navigation":325,"ogImage":44304,"path":44307,"seo":44308,"stem":44309,"__hash__":44310},"content/blogs/2. PPO学习笔记.md","零基础速通 PPO 学习笔记",{"type":7,"value":13032,"toc":44288},[13033,13036,13039,13042,13550,13853,14302,14305,14308,14370,14709,18701,19302,19821,19825,19900,20564,20808,20927,21255,21258,21944,23083,23185,23188,25820,25926,27798,27802,27805,27839,30621,30624,30627,30630,32117,32120,32124,32127,32130,32265,33929,33932,33936,34579,34770,39295,39298,40343,40346,40351,40387,41678,41682,41685,42294,44278,44281],[10,13034,13035],{"id":13035},"学习背景",[14,13037,13038],{},"最近陶瓷到了港科的一个 RA，组里在做 Agent 强化学习相关内容，听从学长的指导来学习 PPO 算法相关知识。",[10,13040,13041],{"id":13041},"符号定义",[14,13043,13044,13045,13117,13118,13147,13148,13219,13220,13248,13249,13278,13279,13416,13417,13488,13489,13517,13518,13549],{},"在强化学习中，",[47,13046,13048,13066],{"className":13047},[451],[47,13049,13051],{"className":13050},[455],[457,13052,13053],{"xmlns":459},[461,13054,13055,13063],{},[464,13056,13057],{},[477,13058,13059,13061],{},[480,13060,285],{},[480,13062,1166],{},[487,13064,13065],{"encoding":489},"a_t",[47,13067,13069],{"className":13068,"ariaHidden":495},[494],[47,13070,13072,13076],{"className":13071},[499],[47,13073],{"className":13074,"style":13075},[503],"height:0.5806em;vertical-align:-0.15em;",[47,13077,13079,13082],{"className":13078},[508],[47,13080,285],{"className":13081},[508,581],[47,13083,13085],{"className":13084},[585],[47,13086,13088,13109],{"className":13087},[520,521],[47,13089,13091,13106],{"className":13090},[525],[47,13092,13095],{"className":13093,"style":13094},[529],"height:0.2806em;",[47,13096,13097,13100],{"style":1638},[47,13098],{"className":13099,"style":929},[537],[47,13101,13103],{"className":13102},[542,543,544,545],[47,13104,1166],{"className":13105},[508,581,545],[47,13107,616],{"className":13108},[615],[47,13110,13112],{"className":13111},[525],[47,13113,13115],{"className":13114,"style":1657},[529],[47,13116],{}," 一般表示 ",[47,13119,13121,13134],{"className":13120},[451],[47,13122,13124],{"className":13123},[455],[457,13125,13126],{"xmlns":459},[461,13127,13128,13132],{},[464,13129,13130],{},[480,13131,1166],{},[487,13133,1166],{"encoding":489},[47,13135,13137],{"className":13136,"ariaHidden":495},[494],[47,13138,13140,13144],{"className":13139},[499],[47,13141],{"className":13142,"style":13143},[503],"height:0.6151em;",[47,13145,1166],{"className":13146},[508,581]," 时刻 Agent 采取的动作（action）；",[47,13149,13151,13170],{"className":13150},[451],[47,13152,13154],{"className":13153},[455],[457,13155,13156],{"xmlns":459},[461,13157,13158,13167],{},[464,13159,13160],{},[477,13161,13162,13165],{},[480,13163,13164],{},"s",[480,13166,1166],{},[487,13168,13169],{"encoding":489},"s_t",[47,13171,13173],{"className":13172,"ariaHidden":495},[494],[47,13174,13176,13179],{"className":13175},[499],[47,13177],{"className":13178,"style":13075},[503],[47,13180,13182,13185],{"className":13181},[508],[47,13183,13164],{"className":13184},[508,581],[47,13186,13188],{"className":13187},[585],[47,13189,13191,13211],{"className":13190},[520,521],[47,13192,13194,13208],{"className":13193},[525],[47,13195,13197],{"className":13196,"style":13094},[529],[47,13198,13199,13202],{"style":1638},[47,13200],{"className":13201,"style":929},[537],[47,13203,13205],{"className":13204},[542,543,544,545],[47,13206,1166],{"className":13207},[508,581,545],[47,13209,616],{"className":13210},[615],[47,13212,13214],{"className":13213},[525],[47,13215,13217],{"className":13216,"style":1657},[529],[47,13218],{}," 表示 ",[47,13221,13223,13236],{"className":13222},[451],[47,13224,13226],{"className":13225},[455],[457,13227,13228],{"xmlns":459},[461,13229,13230,13234],{},[464,13231,13232],{},[480,13233,1166],{},[487,13235,1166],{"encoding":489},[47,13237,13239],{"className":13238,"ariaHidden":495},[494],[47,13240,13242,13245],{"className":13241},[499],[47,13243],{"className":13244,"style":13143},[503],[47,13246,1166],{"className":13247},[508,581]," 时刻 Agent 所处的状态（state）；",[47,13250,13252,13266],{"className":13251},[451],[47,13253,13255],{"className":13254},[455],[457,13256,13257],{"xmlns":459},[461,13258,13259,13263],{},[464,13260,13261],{},[480,13262,7112],{},[487,13264,13265],{"encoding":489},"\\pi",[47,13267,13269],{"className":13268,"ariaHidden":495},[494],[47,13270,13272,13275],{"className":13271},[499],[47,13273],{"className":13274,"style":736},[503],[47,13276,7112],{"className":13277,"style":830},[508,581]," 表示 Agent的策略函数（policy），输入状态，输入采取每个动作的概率分布 ",[47,13280,13282,13315],{"className":13281},[451],[47,13283,13285],{"className":13284},[455],[457,13286,13287],{"xmlns":459},[461,13288,13289,13312],{},[464,13290,13291,13293,13295,13301,13304,13310],{},[480,13292,7112],{},[850,13294,1187],{"stretchy":1186},[477,13296,13297,13299],{},[480,13298,285],{},[480,13300,1166],{},[480,13302,13303],{"mathvariant":1206},"∣",[477,13305,13306,13308],{},[480,13307,13164],{},[480,13309,1166],{},[850,13311,1201],{"stretchy":1186},[487,13313,13314],{"encoding":489},"\\pi(a_t|s_t)",[47,13316,13318],{"className":13317,"ariaHidden":495},[494],[47,13319,13321,13324,13327,13330,13370,13373,13413],{"className":13320},[499],[47,13322],{"className":13323,"style":1264},[503],[47,13325,7112],{"className":13326,"style":830},[508,581],[47,13328,1187],{"className":13329},[512],[47,13331,13333,13336],{"className":13332},[508],[47,13334,285],{"className":13335},[508,581],[47,13337,13339],{"className":13338},[585],[47,13340,13342,13362],{"className":13341},[520,521],[47,13343,13345,13359],{"className":13344},[525],[47,13346,13348],{"className":13347,"style":13094},[529],[47,13349,13350,13353],{"style":1638},[47,13351],{"className":13352,"style":929},[537],[47,13354,13356],{"className":13355},[542,543,544,545],[47,13357,1166],{"className":13358},[508,581,545],[47,13360,616],{"className":13361},[615],[47,13363,13365],{"className":13364},[525],[47,13366,13368],{"className":13367,"style":1657},[529],[47,13369],{},[47,13371,13303],{"className":13372},[508],[47,13374,13376,13379],{"className":13375},[508],[47,13377,13164],{"className":13378},[508,581],[47,13380,13382],{"className":13381},[585],[47,13383,13385,13405],{"className":13384},[520,521],[47,13386,13388,13402],{"className":13387},[525],[47,13389,13391],{"className":13390,"style":13094},[529],[47,13392,13393,13396],{"style":1638},[47,13394],{"className":13395,"style":929},[537],[47,13397,13399],{"className":13398},[542,543,544,545],[47,13400,1166],{"className":13401},[508,581,545],[47,13403,616],{"className":13404},[615],[47,13406,13408],{"className":13407},[525],[47,13409,13411],{"className":13410,"style":1657},[529],[47,13412],{},[47,13414,1201],{"className":13415},[702],"；",[47,13418,13420,13438],{"className":13419},[451],[47,13421,13423],{"className":13422},[455],[457,13424,13425],{"xmlns":459},[461,13426,13427,13435],{},[464,13428,13429],{},[477,13430,13431,13433],{},[480,13432,8196],{},[480,13434,1166],{},[487,13436,13437],{"encoding":489},"r_t",[47,13439,13441],{"className":13440,"ariaHidden":495},[494],[47,13442,13444,13447],{"className":13443},[499],[47,13445],{"className":13446,"style":13075},[503],[47,13448,13450,13453],{"className":13449},[508],[47,13451,8196],{"className":13452,"style":3580},[508,581],[47,13454,13456],{"className":13455},[585],[47,13457,13459,13480],{"className":13458},[520,521],[47,13460,13462,13477],{"className":13461},[525],[47,13463,13465],{"className":13464,"style":13094},[529],[47,13466,13468,13471],{"style":13467},"top:-2.55em;margin-left:-0.0278em;margin-right:0.05em;",[47,13469],{"className":13470,"style":929},[537],[47,13472,13474],{"className":13473},[542,543,544,545],[47,13475,1166],{"className":13476},[508,581,545],[47,13478,616],{"className":13479},[615],[47,13481,13483],{"className":13482},[525],[47,13484,13486],{"className":13485,"style":1657},[529],[47,13487],{}," 一边表示 ",[47,13490,13492,13505],{"className":13491},[451],[47,13493,13495],{"className":13494},[455],[457,13496,13497],{"xmlns":459},[461,13498,13499,13503],{},[464,13500,13501],{},[480,13502,1166],{},[487,13504,1166],{"encoding":489},[47,13506,13508],{"className":13507,"ariaHidden":495},[494],[47,13509,13511,13514],{"className":13510},[499],[47,13512],{"className":13513,"style":13143},[503],[47,13515,1166],{"className":13516},[508,581]," 时刻 Agent 采取行动获得的奖励（Reward）；",[47,13519,13521,13536],{"className":13520},[451],[47,13522,13524],{"className":13523},[455],[457,13525,13526],{"xmlns":459},[461,13527,13528,13533],{},[464,13529,13530],{},[480,13531,13532],{},"τ",[487,13534,13535],{"encoding":489},"\\tau",[47,13537,13539],{"className":13538,"ariaHidden":495},[494],[47,13540,13542,13545],{"className":13541},[499],[47,13543],{"className":13544,"style":736},[503],[47,13546,13532],{"className":13547,"style":13548},[508,581],"margin-right:0.1132em;"," 表示轨迹（Trajectory），Episode 指一次完整的交互过程：从环境开始，到终止结束。Rollout 指按某个策略实际“跑出来”的一段轨迹数据。它不一定非得是完整的一局，也可以只是中间截取的一段。",[14,13551,13552],{},[47,13553,13555,13610],{"className":13554},[451],[47,13556,13558],{"className":13557},[455],[457,13559,13560],{"xmlns":459},[461,13561,13562,13607],{},[464,13563,13564,13566,13572,13574,13580,13582,13588,13590,13596,13598,13605],{},[850,13565,1187],{"stretchy":1186},[477,13567,13568,13570],{},[480,13569,13164],{},[470,13571,2141],{},[850,13573,1192],{"separator":495},[477,13575,13576,13578],{},[480,13577,285],{},[470,13579,2141],{},[850,13581,1192],{"separator":495},[477,13583,13584,13586],{},[480,13585,13164],{},[470,13587,472],{},[850,13589,1192],{"separator":495},[477,13591,13592,13594],{},[480,13593,285],{},[470,13595,472],{},[850,13597,3340],{},[477,13599,13600,13602],{},[480,13601,13164],{},[480,13603,13604],{},"T",[850,13606,1201],{"stretchy":1186},[487,13608,13609],{"encoding":489},"(s_0,a_0,s_1,a_1 \\dots s_T)",[47,13611,13613],{"className":13612,"ariaHidden":495},[494],[47,13614,13616,13619,13622,13662,13665,13668,13708,13711,13714,13754,13757,13760,13800,13803,13806,13809,13850],{"className":13615},[499],[47,13617],{"className":13618,"style":1264},[503],[47,13620,1187],{"className":13621},[512],[47,13623,13625,13628],{"className":13624},[508],[47,13626,13164],{"className":13627},[508,581],[47,13629,13631],{"className":13630},[585],[47,13632,13634,13654],{"className":13633},[520,521],[47,13635,13637,13651],{"className":13636},[525],[47,13638,13640],{"className":13639,"style":3461},[529],[47,13641,13642,13645],{"style":1638},[47,13643],{"className":13644,"style":929},[537],[47,13646,13648],{"className":13647},[542,543,544,545],[47,13649,2141],{"className":13650},[508,545],[47,13652,616],{"className":13653},[615],[47,13655,13657],{"className":13656},[525],[47,13658,13660],{"className":13659,"style":1657},[529],[47,13661],{},[47,13663,1192],{"className":13664},[1299],[47,13666],{"className":13667,"style":1303},[888],[47,13669,13671,13674],{"className":13670},[508],[47,13672,285],{"className":13673},[508,581],[47,13675,13677],{"className":13676},[585],[47,13678,13680,13700],{"className":13679},[520,521],[47,13681,13683,13697],{"className":13682},[525],[47,13684,13686],{"className":13685,"style":3461},[529],[47,13687,13688,13691],{"style":1638},[47,13689],{"className":13690,"style":929},[537],[47,13692,13694],{"className":13693},[542,543,544,545],[47,13695,2141],{"className":13696},[508,545],[47,13698,616],{"className":13699},[615],[47,13701,13703],{"className":13702},[525],[47,13704,13706],{"className":13705,"style":1657},[529],[47,13707],{},[47,13709,1192],{"className":13710},[1299],[47,13712],{"className":13713,"style":1303},[888],[47,13715,13717,13720],{"className":13716},[508],[47,13718,13164],{"className":13719},[508,581],[47,13721,13723],{"className":13722},[585],[47,13724,13726,13746],{"className":13725},[520,521],[47,13727,13729,13743],{"className":13728},[525],[47,13730,13732],{"className":13731,"style":3461},[529],[47,13733,13734,13737],{"style":1638},[47,13735],{"className":13736,"style":929},[537],[47,13738,13740],{"className":13739},[542,543,544,545],[47,13741,472],{"className":13742},[508,545],[47,13744,616],{"className":13745},[615],[47,13747,13749],{"className":13748},[525],[47,13750,13752],{"className":13751,"style":1657},[529],[47,13753],{},[47,13755,1192],{"className":13756},[1299],[47,13758],{"className":13759,"style":1303},[888],[47,13761,13763,13766],{"className":13762},[508],[47,13764,285],{"className":13765},[508,581],[47,13767,13769],{"className":13768},[585],[47,13770,13772,13792],{"className":13771},[520,521],[47,13773,13775,13789],{"className":13774},[525],[47,13776,13778],{"className":13777,"style":3461},[529],[47,13779,13780,13783],{"style":1638},[47,13781],{"className":13782,"style":929},[537],[47,13784,13786],{"className":13785},[542,543,544,545],[47,13787,472],{"className":13788},[508,545],[47,13790,616],{"className":13791},[615],[47,13793,13795],{"className":13794},[525],[47,13796,13798],{"className":13797,"style":1657},[529],[47,13799],{},[47,13801],{"className":13802,"style":1303},[888],[47,13804,3340],{"className":13805},[1349],[47,13807],{"className":13808,"style":1303},[888],[47,13810,13812,13815],{"className":13811},[508],[47,13813,13164],{"className":13814},[508,581],[47,13816,13818],{"className":13817},[585],[47,13819,13821,13842],{"className":13820},[520,521],[47,13822,13824,13839],{"className":13823},[525],[47,13825,13828],{"className":13826,"style":13827},[529],"height:0.3283em;",[47,13829,13830,13833],{"style":1638},[47,13831],{"className":13832,"style":929},[537],[47,13834,13836],{"className":13835},[542,543,544,545],[47,13837,13604],{"className":13838,"style":3555},[508,581,545],[47,13840,616],{"className":13841},[615],[47,13843,13845],{"className":13844},[525],[47,13846,13848],{"className":13847,"style":1657},[529],[47,13849],{},[47,13851,1201],{"className":13852},[702],[14,13854,13855,14074,14075,703],{},[47,13856,13858,13904],{"className":13857},[451],[47,13859,13861],{"className":13860},[455],[457,13862,13863],{"xmlns":459},[461,13864,13865,13901],{},[464,13866,13867,13879,13881,13883,13885,13891,13893,13899],{},[477,13868,13869,13871],{},[480,13870,13164],{},[464,13872,13873,13875,13877],{},[480,13874,1166],{},[850,13876,6147],{},[470,13878,472],{},[850,13880,82],{},[480,13882,12022],{},[850,13884,1187],{"stretchy":1186},[477,13886,13887,13889],{},[480,13888,13164],{},[480,13890,1166],{},[850,13892,1192],{"separator":495},[477,13894,13895,13897],{},[480,13896,285],{},[480,13898,1166],{},[850,13900,1201],{"stretchy":1186},[487,13902,13903],{"encoding":489}," s_{t+1} = f(s_t,a_t)",[47,13905,13907,13972],{"className":13906,"ariaHidden":495},[494],[47,13908,13910,13914,13963,13966,13969],{"className":13909},[499],[47,13911],{"className":13912,"style":13913},[503],"height:0.6389em;vertical-align:-0.2083em;",[47,13915,13917,13920],{"className":13916},[508],[47,13918,13164],{"className":13919},[508,581],[47,13921,13923],{"className":13922},[585],[47,13924,13926,13955],{"className":13925},[520,521],[47,13927,13929,13952],{"className":13928},[525],[47,13930,13932],{"className":13931,"style":3461},[529],[47,13933,13934,13937],{"style":1638},[47,13935],{"className":13936,"style":929},[537],[47,13938,13940],{"className":13939},[542,543,544,545],[47,13941,13943,13946,13949],{"className":13942},[508,545],[47,13944,1166],{"className":13945},[508,581,545],[47,13947,6147],{"className":13948},[942,545],[47,13950,472],{"className":13951},[508,545],[47,13953,616],{"className":13954},[615],[47,13956,13958],{"className":13957},[525],[47,13959,13961],{"className":13960,"style":7308},[529],[47,13962],{},[47,13964],{"className":13965,"style":889},[888],[47,13967,82],{"className":13968},[893],[47,13970],{"className":13971,"style":889},[888],[47,13973,13975,13978,13982,13985,14025,14028,14031,14071],{"className":13974},[499],[47,13976],{"className":13977,"style":1264},[503],[47,13979,12022],{"className":13980,"style":13981},[508,581],"margin-right:0.10764em;",[47,13983,1187],{"className":13984},[512],[47,13986,13988,13991],{"className":13987},[508],[47,13989,13164],{"className":13990},[508,581],[47,13992,13994],{"className":13993},[585],[47,13995,13997,14017],{"className":13996},[520,521],[47,13998,14000,14014],{"className":13999},[525],[47,14001,14003],{"className":14002,"style":13094},[529],[47,14004,14005,14008],{"style":1638},[47,14006],{"className":14007,"style":929},[537],[47,14009,14011],{"className":14010},[542,543,544,545],[47,14012,1166],{"className":14013},[508,581,545],[47,14015,616],{"className":14016},[615],[47,14018,14020],{"className":14019},[525],[47,14021,14023],{"className":14022,"style":1657},[529],[47,14024],{},[47,14026,1192],{"className":14027},[1299],[47,14029],{"className":14030,"style":1303},[888],[47,14032,14034,14037],{"className":14033},[508],[47,14035,285],{"className":14036},[508,581],[47,14038,14040],{"className":14039},[585],[47,14041,14043,14063],{"className":14042},[520,521],[47,14044,14046,14060],{"className":14045},[525],[47,14047,14049],{"className":14048,"style":13094},[529],[47,14050,14051,14054],{"style":1638},[47,14052],{"className":14053,"style":929},[537],[47,14055,14057],{"className":14056},[542,543,544,545],[47,14058,1166],{"className":14059},[508,581,545],[47,14061,616],{"className":14062},[615],[47,14064,14066],{"className":14065},[525],[47,14067,14069],{"className":14068,"style":1657},[529],[47,14070],{},[47,14072,1201],{"className":14073},[702]," 或者 ",[47,14076,14078,14128],{"className":14077},[451],[47,14079,14081],{"className":14080},[455],[457,14082,14083],{"xmlns":459},[461,14084,14085,14125],{},[464,14086,14087,14099,14101,14103,14105,14107,14109,14115,14117,14123],{},[477,14088,14089,14091],{},[480,14090,13164],{},[464,14092,14093,14095,14097],{},[480,14094,1166],{},[850,14096,6147],{},[470,14098,472],{},[850,14100,82],{},[480,14102,5974],{},[850,14104,1187],{"stretchy":1186},[850,14106,2203],{},[480,14108,13303],{"mathvariant":1206},[477,14110,14111,14113],{},[480,14112,13164],{},[480,14114,1166],{},[850,14116,1192],{"separator":495},[477,14118,14119,14121],{},[480,14120,285],{},[480,14122,1166],{},[850,14124,1201],{"stretchy":1186},[487,14126,14127],{"encoding":489},"s_{t+1} = P(\\cdot|s_t,a_t)",[47,14129,14131,14195],{"className":14130,"ariaHidden":495},[494],[47,14132,14134,14137,14186,14189,14192],{"className":14133},[499],[47,14135],{"className":14136,"style":13913},[503],[47,14138,14140,14143],{"className":14139},[508],[47,14141,13164],{"className":14142},[508,581],[47,14144,14146],{"className":14145},[585],[47,14147,14149,14178],{"className":14148},[520,521],[47,14150,14152,14175],{"className":14151},[525],[47,14153,14155],{"className":14154,"style":3461},[529],[47,14156,14157,14160],{"style":1638},[47,14158],{"className":14159,"style":929},[537],[47,14161,14163],{"className":14162},[542,543,544,545],[47,14164,14166,14169,14172],{"className":14165},[508,545],[47,14167,1166],{"className":14168},[508,581,545],[47,14170,6147],{"className":14171},[942,545],[47,14173,472],{"className":14174},[508,545],[47,14176,616],{"className":14177},[615],[47,14179,14181],{"className":14180},[525],[47,14182,14184],{"className":14183,"style":7308},[529],[47,14185],{},[47,14187],{"className":14188,"style":889},[888],[47,14190,82],{"className":14191},[893],[47,14193],{"className":14194,"style":889},[888],[47,14196,14198,14201,14204,14207,14210,14213,14253,14256,14259,14299],{"className":14197},[499],[47,14199],{"className":14200,"style":1264},[503],[47,14202,5974],{"className":14203,"style":3555},[508,581],[47,14205,1187],{"className":14206},[512],[47,14208,2203],{"className":14209},[508],[47,14211,13303],{"className":14212},[508],[47,14214,14216,14219],{"className":14215},[508],[47,14217,13164],{"className":14218},[508,581],[47,14220,14222],{"className":14221},[585],[47,14223,14225,14245],{"className":14224},[520,521],[47,14226,14228,14242],{"className":14227},[525],[47,14229,14231],{"className":14230,"style":13094},[529],[47,14232,14233,14236],{"style":1638},[47,14234],{"className":14235,"style":929},[537],[47,14237,14239],{"className":14238},[542,543,544,545],[47,14240,1166],{"className":14241},[508,581,545],[47,14243,616],{"className":14244},[615],[47,14246,14248],{"className":14247},[525],[47,14249,14251],{"className":14250,"style":1657},[529],[47,14252],{},[47,14254,1192],{"className":14255},[1299],[47,14257],{"className":14258,"style":1303},[888],[47,14260,14262,14265],{"className":14261},[508],[47,14263,285],{"className":14264},[508,581],[47,14266,14268],{"className":14267},[585],[47,14269,14271,14291],{"className":14270},[520,521],[47,14272,14274,14288],{"className":14273},[525],[47,14275,14277],{"className":14276,"style":13094},[529],[47,14278,14279,14282],{"style":1638},[47,14280],{"className":14281,"style":929},[537],[47,14283,14285],{"className":14284},[542,543,544,545],[47,14286,1166],{"className":14287},[508,581,545],[47,14289,616],{"className":14290},[615],[47,14292,14294],{"className":14293},[525],[47,14295,14297],{"className":14296,"style":1657},[529],[47,14298],{},[47,14300,1201],{"className":14301},[702],[14,14303,14304],{},"Return 一般表示回报，指从当前时间点到游戏结束时奖励的累积和或者加权累积和。",[10,14306,14307],{"id":14307},"训练目标",[14,14309,14310,14311,14339,14340,14369],{},"训练一个 Policy 神经网络 ",[47,14312,14314,14327],{"className":14313},[451],[47,14315,14317],{"className":14316},[455],[457,14318,14319],{"xmlns":459},[461,14320,14321,14325],{},[464,14322,14323],{},[480,14324,7112],{},[487,14326,13265],{"encoding":489},[47,14328,14330],{"className":14329,"ariaHidden":495},[494],[47,14331,14333,14336],{"className":14332},[499],[47,14334],{"className":14335,"style":736},[503],[47,14337,7112],{"className":14338,"style":830},[508,581],"，在所有状态 ",[47,14341,14343,14356],{"className":14342},[451],[47,14344,14346],{"className":14345},[455],[457,14347,14348],{"xmlns":459},[461,14349,14350,14354],{},[464,14351,14352],{},[480,14353,8215],{},[487,14355,8215],{"encoding":489},[47,14357,14359],{"className":14358,"ariaHidden":495},[494],[47,14360,14362,14366],{"className":14361},[499],[47,14363],{"className":14364,"style":14365},[503],"height:0.6833em;",[47,14367,8215],{"className":14368,"style":6420},[508,581]," 下，给出相应的 Action，得到的 Return 的期望最大。",[14,14371,14372],{},[47,14373,14375,14452],{"className":14374},[451],[47,14376,14378],{"className":14377},[455],[457,14379,14380],{"xmlns":459},[461,14381,14382,14449],{},[464,14383,14384,14386,14388,14390,14392,14394,14396,14420,14422,14429,14431,14433,14435,14437,14443,14445,14447],{},[480,14385,5977],{},[850,14387,1187],{"stretchy":1186},[480,14389,859],{},[850,14391,1187],{"stretchy":1186},[480,14393,13532],{},[850,14395,1201],{"stretchy":1186},[477,14397,14398,14400],{},[850,14399,1201],{"stretchy":1186},[464,14401,14402,14404,14407,14414,14416,14418],{},[480,14403,13532],{},[850,14405,14406],{},"∼",[477,14408,14409,14411],{},[480,14410,5974],{},[480,14412,14413],{},"θ",[850,14415,1187],{"stretchy":1186},[480,14417,13532],{},[850,14419,1201],{"stretchy":1186},[850,14421,82],{},[14423,14424,14425,14427],"munder",{},[850,14426,2213],{},[480,14428,13532],{},[480,14430,859],{},[850,14432,1187],{"stretchy":1186},[480,14434,13532],{},[850,14436,1201],{"stretchy":1186},[477,14438,14439,14441],{},[480,14440,5974],{},[480,14442,14413],{},[850,14444,1187],{"stretchy":1186},[480,14446,13532],{},[850,14448,1201],{"stretchy":1186},[487,14450,14451],{"encoding":489}," E (R(\\tau))_{\\tau \\sim P_{\\theta}(\\tau)} = \\sum\\limits_\\tau R(\\tau)P_\\theta(\\tau)",[47,14453,14455,14590],{"className":14454,"ariaHidden":495},[494],[47,14456,14458,14462,14465,14468,14472,14475,14478,14481,14581,14584,14587],{"className":14457},[499],[47,14459],{"className":14460,"style":14461},[503],"height:1.1052em;vertical-align:-0.3552em;",[47,14463,5977],{"className":14464,"style":6420},[508,581],[47,14466,1187],{"className":14467},[512],[47,14469,859],{"className":14470,"style":14471},[508,581],"margin-right:0.00773em;",[47,14473,1187],{"className":14474},[512],[47,14476,13532],{"className":14477,"style":13548},[508,581],[47,14479,1201],{"className":14480},[702],[47,14482,14484,14487],{"className":14483},[702],[47,14485,1201],{"className":14486},[702],[47,14488,14490],{"className":14489},[585],[47,14491,14493,14573],{"className":14492},[520,521],[47,14494,14496,14570],{"className":14495},[525],[47,14497,14499],{"className":14498,"style":595},[529],[47,14500,14502,14505],{"style":14501},"top:-2.5198em;margin-left:0em;margin-right:0.05em;",[47,14503],{"className":14504,"style":929},[537],[47,14506,14508],{"className":14507},[542,543,544,545],[47,14509,14511,14514,14517,14561,14564,14567],{"className":14510},[508,545],[47,14512,13532],{"className":14513,"style":13548},[508,581,545],[47,14515,14406],{"className":14516},[893,545],[47,14518,14520,14523],{"className":14519},[508,545],[47,14521,5974],{"className":14522,"style":3555},[508,581,545],[47,14524,14526],{"className":14525},[585],[47,14527,14529,14553],{"className":14528},[520,521],[47,14530,14532,14550],{"className":14531},[525],[47,14533,14535],{"className":14534,"style":595},[529],[47,14536,14538,14541],{"style":14537},"top:-2.3488em;margin-left:-0.1389em;margin-right:0.0714em;",[47,14539],{"className":14540,"style":602},[537],[47,14542,14544],{"className":14543},[542,606,607,545],[47,14545,14547],{"className":14546},[508,545],[47,14548,14413],{"className":14549,"style":3580},[508,581,545],[47,14551,616],{"className":14552},[615],[47,14554,14556],{"className":14555},[525],[47,14557,14559],{"className":14558,"style":623},[529],[47,14560],{},[47,14562,1187],{"className":14563},[512,545],[47,14565,13532],{"className":14566,"style":13548},[508,581,545],[47,14568,1201],{"className":14569},[702,545],[47,14571,616],{"className":14572},[615],[47,14574,14576],{"className":14575},[525],[47,14577,14579],{"className":14578,"style":6472},[529],[47,14580],{},[47,14582],{"className":14583,"style":889},[888],[47,14585,82],{"className":14586},[893],[47,14588],{"className":14589,"style":889},[888],[47,14591,14593,14597,14645,14648,14651,14654,14657,14660,14700,14703,14706],{"className":14592},[499],[47,14594],{"className":14595,"style":14596},[503],"height:1.7em;vertical-align:-0.95em;",[47,14598,14601],{"className":14599},[1338,14600],"op-limits",[47,14602,14604,14636],{"className":14603},[520,521],[47,14605,14607,14633],{"className":14606},[525],[47,14608,14611,14623],{"className":14609,"style":14610},[529],"height:0.75em;",[47,14612,14614,14617],{"style":14613},"top:-2.15em;margin-left:0em;",[47,14615],{"className":14616,"style":538},[537],[47,14618,14620],{"className":14619},[542,543,544,545],[47,14621,13532],{"className":14622,"style":13548},[508,581,545],[47,14624,14625,14628],{"style":567},[47,14626],{"className":14627,"style":538},[537],[47,14629,14630],{},[47,14631,2213],{"className":14632},[1338,2297,2298],[47,14634,616],{"className":14635},[615],[47,14637,14639],{"className":14638},[525],[47,14640,14643],{"className":14641,"style":14642},[529],"height:0.95em;",[47,14644],{},[47,14646],{"className":14647,"style":1303},[888],[47,14649,859],{"className":14650,"style":14471},[508,581],[47,14652,1187],{"className":14653},[512],[47,14655,13532],{"className":14656,"style":13548},[508,581],[47,14658,1201],{"className":14659},[702],[47,14661,14663,14666],{"className":14662},[508],[47,14664,5974],{"className":14665,"style":3555},[508,581],[47,14667,14669],{"className":14668},[585],[47,14670,14672,14692],{"className":14671},[520,521],[47,14673,14675,14689],{"className":14674},[525],[47,14676,14678],{"className":14677,"style":1635},[529],[47,14679,14680,14683],{"style":7830},[47,14681],{"className":14682,"style":929},[537],[47,14684,14686],{"className":14685},[542,543,544,545],[47,14687,14413],{"className":14688,"style":3580},[508,581,545],[47,14690,616],{"className":14691},[615],[47,14693,14695],{"className":14694},[525],[47,14696,14698],{"className":14697,"style":1657},[529],[47,14699],{},[47,14701,1187],{"className":14702},[512],[47,14704,13532],{"className":14705,"style":13548},[508,581],[47,14707,1201],{"className":14708},[702],[47,14710,14713],{"className":14711},[14712],"katex-display",[47,14714,14716,15496],{"className":14715},[451],[47,14717,14719],{"className":14718},[455],[457,14720,14721],{"xmlns":459,"display":9848},[461,14722,14723,15493],{},[14724,14725,14729,14815,14861,14941,15019,15113,15186,15289,15391],"mtable",{"rowspacing":14726,"columnalign":14727,"columnspacing":14728},"0.25em","right left","0em",[14730,14731,14732,14777],"mtr",{},[14733,14734,14735],"mtd",{},[14736,14737,14738],"mstyle",{"scriptlevel":2141,"displaystyle":495},[464,14739,14740,14743,14745,14747,14749,14751,14753,14755],{},[480,14741,14742],{"mathvariant":1206},"∇",[480,14744,5977],{},[850,14746,1187],{"stretchy":1186},[480,14748,859],{},[850,14750,1187],{"stretchy":1186},[480,14752,13532],{},[850,14754,1201],{"stretchy":1186},[477,14756,14757,14759],{},[850,14758,1201],{"stretchy":1186},[464,14760,14761,14763,14765,14771,14773,14775],{},[480,14762,13532],{},[850,14764,14406],{},[477,14766,14767,14769],{},[480,14768,5974],{},[480,14770,14413],{},[850,14772,1187],{"stretchy":1186},[480,14774,13532],{},[850,14776,1201],{"stretchy":1186},[14733,14778,14779],{},[14736,14780,14781],{"scriptlevel":2141,"displaystyle":495},[464,14782,14783,14785,14787,14789,14795,14797,14799,14801,14803,14809,14811,14813],{},[464,14784],{},[850,14786,82],{},[480,14788,14742],{"mathvariant":1206},[14423,14790,14791,14793],{},[850,14792,2213],{},[480,14794,13532],{},[480,14796,859],{},[850,14798,1187],{"stretchy":1186},[480,14800,13532],{},[850,14802,1201],{"stretchy":1186},[477,14804,14805,14807],{},[480,14806,5974],{},[480,14808,14413],{},[850,14810,1187],{"stretchy":1186},[480,14812,13532],{},[850,14814,1201],{"stretchy":1186},[14730,14816,14817,14823],{},[14733,14818,14819],{},[14736,14820,14821],{"scriptlevel":2141,"displaystyle":495},[464,14822],{},[14733,14824,14825],{},[14736,14826,14827],{"scriptlevel":2141,"displaystyle":495},[464,14828,14829,14831,14833,14839,14841,14843,14845,14847,14849,14855,14857,14859],{},[464,14830],{},[850,14832,82],{},[14423,14834,14835,14837],{},[850,14836,2213],{},[480,14838,13532],{},[480,14840,859],{},[850,14842,1187],{"stretchy":1186},[480,14844,13532],{},[850,14846,1201],{"stretchy":1186},[480,14848,14742],{"mathvariant":1206},[477,14850,14851,14853],{},[480,14852,5974],{},[480,14854,14413],{},[850,14856,1187],{"stretchy":1186},[480,14858,13532],{},[850,14860,1201],{"stretchy":1186},[14730,14862,14863,14869],{},[14733,14864,14865],{},[14736,14866,14867],{"scriptlevel":2141,"displaystyle":495},[464,14868],{},[14733,14870,14871],{},[14736,14872,14873],{"scriptlevel":2141,"displaystyle":495},[464,14874,14875,14877,14879,14885,14887,14889,14891,14893,14895,14901,14903,14905,14907,14909],{},[464,14876],{},[850,14878,82],{},[14423,14880,14881,14883],{},[850,14882,2213],{},[480,14884,13532],{},[480,14886,859],{},[850,14888,1187],{"stretchy":1186},[480,14890,13532],{},[850,14892,1201],{"stretchy":1186},[480,14894,14742],{"mathvariant":1206},[477,14896,14897,14899],{},[480,14898,5974],{},[480,14900,14413],{},[850,14902,1187],{"stretchy":1186},[480,14904,13532],{},[850,14906,1201],{"stretchy":1186},[850,14908,2203],{},[14736,14910,14911],{"scriptlevel":2141,"displaystyle":495},[467,14912,14913,14927],{},[464,14914,14915,14921,14923,14925],{},[477,14916,14917,14919],{},[480,14918,5974],{},[480,14920,14413],{},[850,14922,1187],{"stretchy":1186},[480,14924,13532],{},[850,14926,1201],{"stretchy":1186},[464,14928,14929,14935,14937,14939],{},[477,14930,14931,14933],{},[480,14932,5974],{},[480,14934,14413],{},[850,14936,1187],{"stretchy":1186},[480,14938,13532],{},[850,14940,1201],{"stretchy":1186},[14730,14942,14943,14949],{},[14733,14944,14945],{},[14736,14946,14947],{"scriptlevel":2141,"displaystyle":495},[464,14948],{},[14733,14950,14951],{},[14736,14952,14953],{"scriptlevel":2141,"displaystyle":495},[464,14954,14955,14957,14959,14965,14971,14973,14975,14977,14979,14981,14983,14985],{},[464,14956],{},[850,14958,82],{},[14423,14960,14961,14963],{},[850,14962,2213],{},[480,14964,13532],{},[477,14966,14967,14969],{},[480,14968,5974],{},[480,14970,14413],{},[850,14972,1187],{"stretchy":1186},[480,14974,13532],{},[850,14976,1201],{"stretchy":1186},[480,14978,859],{},[850,14980,1187],{"stretchy":1186},[480,14982,13532],{},[850,14984,1201],{"stretchy":1186},[14736,14986,14987],{"scriptlevel":2141,"displaystyle":495},[467,14988,14989,15005],{},[464,14990,14991,14993,14999,15001,15003],{},[480,14992,14742],{"mathvariant":1206},[477,14994,14995,14997],{},[480,14996,5974],{},[480,14998,14413],{},[850,15000,1187],{"stretchy":1186},[480,15002,13532],{},[850,15004,1201],{"stretchy":1186},[464,15006,15007,15013,15015,15017],{},[477,15008,15009,15011],{},[480,15010,5974],{},[480,15012,14413],{},[850,15014,1187],{"stretchy":1186},[480,15016,13532],{},[850,15018,1201],{"stretchy":1186},[14730,15020,15021,15027],{},[14733,15022,15023],{},[14736,15024,15025],{"scriptlevel":2141,"displaystyle":495},[464,15026],{},[14733,15028,15029],{},[14736,15030,15031],{"scriptlevel":2141,"displaystyle":495},[464,15032,15033,15035,15038,15044,15059,15061,15063,15069,15071],{},[464,15034],{},[850,15036,15037],{},"≈",[467,15039,15040,15042],{},[470,15041,472],{},[480,15043,7692],{},[15045,15046,15047,15049,15057],"munderover",{},[850,15048,2213],{},[464,15050,15051,15053,15055],{},[480,15052,724],{},[850,15054,82],{},[470,15056,472],{},[480,15058,7692],{},[480,15060,859],{},[850,15062,1187],{"stretchy":1186},[854,15064,15065,15067],{},[480,15066,13532],{},[480,15068,724],{},[850,15070,1201],{"stretchy":1186},[14736,15072,15073],{"scriptlevel":2141,"displaystyle":495},[467,15074,15075,15095],{},[464,15076,15077,15079,15085,15087,15093],{},[480,15078,14742],{"mathvariant":1206},[477,15080,15081,15083],{},[480,15082,14],{},[480,15084,14413],{},[850,15086,1187],{"stretchy":1186},[854,15088,15089,15091],{},[480,15090,13532],{},[480,15092,724],{},[850,15094,1201],{"stretchy":1186},[464,15096,15097,15103,15105,15111],{},[477,15098,15099,15101],{},[480,15100,5974],{},[480,15102,14413],{},[850,15104,1187],{"stretchy":1186},[854,15106,15107,15109],{},[480,15108,13532],{},[480,15110,724],{},[850,15112,1201],{"stretchy":1186},[14730,15114,15115,15121],{},[14733,15116,15117],{},[14736,15118,15119],{"scriptlevel":2141,"displaystyle":495},[464,15120],{},[14733,15122,15123],{},[14736,15124,15125],{"scriptlevel":2141,"displaystyle":495},[464,15126,15127,15129,15131,15137,15151,15153,15155,15161,15163,15165,15168,15170,15176,15178,15184],{},[464,15128],{},[850,15130,82],{},[467,15132,15133,15135],{},[470,15134,472],{},[480,15136,7692],{},[15045,15138,15139,15141,15149],{},[850,15140,2213],{},[464,15142,15143,15145,15147],{},[480,15144,724],{},[850,15146,82],{},[470,15148,472],{},[480,15150,7692],{},[480,15152,859],{},[850,15154,1187],{"stretchy":1186},[854,15156,15157,15159],{},[480,15158,13532],{},[480,15160,724],{},[850,15162,1201],{"stretchy":1186},[480,15164,14742],{"mathvariant":1206},[480,15166,15167],{},"log",[850,15169,1210],{},[477,15171,15172,15174],{},[480,15173,5974],{},[480,15175,14413],{},[850,15177,1187],{"stretchy":1186},[854,15179,15180,15182],{},[480,15181,13532],{},[480,15183,724],{},[850,15185,1201],{"stretchy":1186},[14730,15187,15188,15194],{},[14733,15189,15190],{},[14736,15191,15192],{"scriptlevel":2141,"displaystyle":495},[464,15193],{},[14733,15195,15196],{},[14736,15197,15198],{"scriptlevel":2141,"displaystyle":495},[464,15199,15200,15202,15204,15210,15224,15226,15228,15234,15236,15238,15240,15242,15261,15267,15269,15277,15279,15287],{},[464,15201],{},[850,15203,82],{},[467,15205,15206,15208],{},[470,15207,472],{},[480,15209,7692],{},[15045,15211,15212,15214,15222],{},[850,15213,2213],{},[464,15215,15216,15218,15220],{},[480,15217,724],{},[850,15219,82],{},[470,15221,472],{},[480,15223,7692],{},[480,15225,859],{},[850,15227,1187],{"stretchy":1186},[854,15229,15230,15232],{},[480,15231,13532],{},[480,15233,724],{},[850,15235,1201],{"stretchy":1186},[480,15237,14742],{"mathvariant":1206},[480,15239,15167],{},[850,15241,1210],{},[15045,15243,15244,15247,15255],{},[850,15245,15246],{},"∏",[464,15248,15249,15251,15253],{},[480,15250,1166],{},[850,15252,82],{},[470,15254,472],{},[477,15256,15257,15259],{},[480,15258,13604],{},[480,15260,724],{},[477,15262,15263,15265],{},[480,15264,5974],{},[480,15266,14413],{},[850,15268,1187],{"stretchy":1186},[2209,15270,15271,15273,15275],{},[480,15272,285],{},[480,15274,724],{},[480,15276,1166],{},[480,15278,13303],{"mathvariant":1206},[2209,15280,15281,15283,15285],{},[480,15282,13164],{},[480,15284,724],{},[480,15286,1166],{},[850,15288,1201],{"stretchy":1186},[14730,15290,15291,15297],{},[14733,15292,15293],{},[14736,15294,15295],{"scriptlevel":2141,"displaystyle":495},[464,15296],{},[14733,15298,15299],{},[14736,15300,15301],{"scriptlevel":2141,"displaystyle":495},[464,15302,15303,15305,15307,15313,15327,15329,15331,15337,15339,15357,15359,15361,15363,15369,15371,15379,15381,15389],{},[464,15304],{},[850,15306,82],{},[467,15308,15309,15311],{},[470,15310,472],{},[480,15312,7692],{},[15045,15314,15315,15317,15325],{},[850,15316,2213],{},[464,15318,15319,15321,15323],{},[480,15320,724],{},[850,15322,82],{},[470,15324,472],{},[480,15326,7692],{},[480,15328,859],{},[850,15330,1187],{"stretchy":1186},[854,15332,15333,15335],{},[480,15334,13532],{},[480,15336,724],{},[850,15338,1201],{"stretchy":1186},[15045,15340,15341,15343,15351],{},[850,15342,2213],{},[464,15344,15345,15347,15349],{},[480,15346,1166],{},[850,15348,82],{},[470,15350,472],{},[477,15352,15353,15355],{},[480,15354,13604],{},[480,15356,724],{},[480,15358,14742],{"mathvariant":1206},[480,15360,15167],{},[850,15362,1210],{},[477,15364,15365,15367],{},[480,15366,5974],{},[480,15368,14413],{},[850,15370,1187],{"stretchy":1186},[2209,15372,15373,15375,15377],{},[480,15374,285],{},[480,15376,724],{},[480,15378,1166],{},[480,15380,13303],{"mathvariant":1206},[2209,15382,15383,15385,15387],{},[480,15384,13164],{},[480,15386,724],{},[480,15388,1166],{},[850,15390,1201],{"stretchy":1186},[14730,15392,15393,15399],{},[14733,15394,15395],{},[14736,15396,15397],{"scriptlevel":2141,"displaystyle":495},[464,15398],{},[14733,15400,15401],{},[14736,15402,15403],{"scriptlevel":2141,"displaystyle":495},[464,15404,15405,15407,15409,15415,15429,15447,15449,15451,15457,15459,15461,15463,15465,15471,15473,15481,15483,15491],{},[464,15406],{},[850,15408,82],{},[467,15410,15411,15413],{},[470,15412,472],{},[480,15414,7692],{},[15045,15416,15417,15419,15427],{},[850,15418,2213],{},[464,15420,15421,15423,15425],{},[480,15422,724],{},[850,15424,82],{},[470,15426,472],{},[480,15428,7692],{},[15045,15430,15431,15433,15441],{},[850,15432,2213],{},[464,15434,15435,15437,15439],{},[480,15436,1166],{},[850,15438,82],{},[470,15440,472],{},[477,15442,15443,15445],{},[480,15444,13604],{},[480,15446,724],{},[480,15448,859],{},[850,15450,1187],{"stretchy":1186},[854,15452,15453,15455],{},[480,15454,13532],{},[480,15456,724],{},[850,15458,1201],{"stretchy":1186},[480,15460,14742],{"mathvariant":1206},[480,15462,15167],{},[850,15464,1210],{},[477,15466,15467,15469],{},[480,15468,5974],{},[480,15470,14413],{},[850,15472,1187],{"stretchy":1186},[2209,15474,15475,15477,15479],{},[480,15476,285],{},[480,15478,724],{},[480,15480,1166],{},[480,15482,13303],{"mathvariant":1206},[2209,15484,15485,15487,15489],{},[480,15486,13164],{},[480,15488,724],{},[480,15490,1166],{},[850,15492,1201],{"stretchy":1186},[487,15494,15495],{"encoding":489},"\\begin{aligned}\n\\nabla E(R(\\tau))_{\\tau \\sim P_\\theta(\\tau)} &= \\nabla \\sum_\\tau R(\\tau)P_\\theta(\\tau)\\\\\n&= \\sum_\\tau R(\\tau)\\nabla P_\\theta(\\tau)\\\\\n&= \\sum_\\tau R(\\tau)\\nabla P_\\theta(\\tau) \\cdot \\dfrac{P_\\theta(\\tau)}{P_\\theta(\\tau)}\\\\\n&= \\sum_\\tau P_\\theta(\\tau)R(\\tau) \\dfrac{\\nabla P_\\theta(\\tau)}{P_\\theta(\\tau)}\\\\\n&\\approx \\frac{1}{N}\\sum\\limits_{n=1}^N R(\\tau^n)\\dfrac{\\nabla p_\\theta(\\tau^n)}{P_\\theta(\\tau^n)}\\\\\n&= \\frac{1}{N}\\sum\\limits_{n=1}^N R(\\tau^n)\\nabla \\log P_\\theta(\\tau^n)\\\\\n&= \\frac{1}{N}\\sum\\limits_{n=1}^N R(\\tau^n)\\nabla \\log \\prod\\limits_{t=1}^{T_n}P_\\theta(a_n^t|s_n^t)\\\\\n&= \\frac{1}{N}\\sum\\limits_{n=1}^N R(\\tau^n)\\sum\\limits_{t=1}^{T_n}\\nabla \\log P_\\theta(a_n^t|s_n^t)\\\\\n&= \\frac{1}{N}\\sum\\limits_{n=1}^N \\sum\\limits_{t=1}^{T_n} R(\\tau^n)\\nabla \\log P_\\theta(a_n^t|s_n^t)\\\\\n\\end{aligned}",[47,15497,15499],{"className":15498,"ariaHidden":495},[494],[47,15500,15502,15506],{"className":15501},[499],[47,15503],{"className":15504,"style":15505},[503],"height:28.1646em;vertical-align:-13.8323em;",[47,15507,15509],{"className":15508},[508],[47,15510,15512,15736],{"className":15511},[14724],[47,15513,15516],{"className":15514},[15515],"col-align-r",[47,15517,15519,15727],{"className":15518},[520,521],[47,15520,15522,15724],{"className":15521},[525],[47,15523,15526,15652,15661,15670,15679,15688,15697,15706,15715],{"className":15524,"style":15525},[529],"height:14.3323em;",[47,15527,15529,15533],{"style":15528},"top:-17.1217em;",[47,15530],{"className":15531,"style":15532},[537],"height:3.8394em;",[47,15534,15536,15539,15542,15545,15548,15551,15554,15557],{"className":15535},[508],[47,15537,14742],{"className":15538},[508],[47,15540,5977],{"className":15541,"style":6420},[508,581],[47,15543,1187],{"className":15544},[512],[47,15546,859],{"className":15547,"style":14471},[508,581],[47,15549,1187],{"className":15550},[512],[47,15552,13532],{"className":15553,"style":13548},[508,581],[47,15555,1201],{"className":15556},[702],[47,15558,15560,15563],{"className":15559},[702],[47,15561,1201],{"className":15562},[702],[47,15564,15566],{"className":15565},[585],[47,15567,15569,15644],{"className":15568},[520,521],[47,15570,15572,15641],{"className":15571},[525],[47,15573,15575],{"className":15574,"style":595},[529],[47,15576,15577,15580],{"style":14501},[47,15578],{"className":15579,"style":929},[537],[47,15581,15583],{"className":15582},[542,543,544,545],[47,15584,15586,15589,15592,15632,15635,15638],{"className":15585},[508,545],[47,15587,13532],{"className":15588,"style":13548},[508,581,545],[47,15590,14406],{"className":15591},[893,545],[47,15593,15595,15598],{"className":15594},[508,545],[47,15596,5974],{"className":15597,"style":3555},[508,581,545],[47,15599,15601],{"className":15600},[585],[47,15602,15604,15624],{"className":15603},[520,521],[47,15605,15607,15621],{"className":15606},[525],[47,15608,15610],{"className":15609,"style":595},[529],[47,15611,15612,15615],{"style":14537},[47,15613],{"className":15614,"style":602},[537],[47,15616,15618],{"className":15617},[542,606,607,545],[47,15619,14413],{"className":15620,"style":3580},[508,581,545],[47,15622,616],{"className":15623},[615],[47,15625,15627],{"className":15626},[525],[47,15628,15630],{"className":15629,"style":623},[529],[47,15631],{},[47,15633,1187],{"className":15634},[512,545],[47,15636,13532],{"className":15637,"style":13548},[508,581,545],[47,15639,1201],{"className":15640},[702,545],[47,15642,616],{"className":15643},[615],[47,15645,15647],{"className":15646},[525],[47,15648,15650],{"className":15649,"style":6472},[529],[47,15651],{},[47,15653,15655,15658],{"style":15654},"top:-14.5217em;",[47,15656],{"className":15657,"style":15532},[537],[47,15659],{"className":15660},[508],[47,15662,15664,15667],{"style":15663},"top:-11.5447em;",[47,15665],{"className":15666,"style":15532},[537],[47,15668],{"className":15669},[508],[47,15671,15673,15676],{"style":15672},"top:-8.5677em;",[47,15674],{"className":15675,"style":15532},[537],[47,15677],{"className":15678},[508],[47,15680,15682,15685],{"style":15681},"top:-5.1894em;",[47,15683],{"className":15684,"style":15532},[537],[47,15686],{"className":15687},[508],[47,15689,15691,15694],{"style":15690},"top:-1.7939em;",[47,15692],{"className":15693,"style":15532},[537],[47,15695],{"className":15696},[508],[47,15698,15700,15703],{"style":15699},"top:1.6126em;",[47,15701],{"className":15702,"style":15532},[537],[47,15704],{"className":15705},[508],[47,15707,15709,15712],{"style":15708},"top:5.0192em;",[47,15710],{"className":15711,"style":15532},[537],[47,15713],{"className":15714},[508],[47,15716,15718,15721],{"style":15717},"top:8.4257em;",[47,15719],{"className":15720,"style":15532},[537],[47,15722],{"className":15723},[508],[47,15725,616],{"className":15726},[615],[47,15728,15730],{"className":15729},[525],[47,15731,15734],{"className":15732,"style":15733},[529],"height:13.8323em;",[47,15735],{},[47,15737,15740],{"className":15738},[15739],"col-align-l",[47,15741,15743,18693],{"className":15742},[520,521],[47,15744,15746,18690],{"className":15745},[525],[47,15747,15749,15889,16020,16318,16603,17013,17295,17763,18228],{"className":15748,"style":15525},[529],[47,15750,15751,15754],{"style":15528},[47,15752],{"className":15753,"style":15532},[537],[47,15755,15757,15760,15763,15766,15769,15772,15775,15825,15828,15831,15834,15837,15840,15880,15883,15886],{"className":15756},[508],[47,15758],{"className":15759},[508],[47,15761],{"className":15762,"style":889},[888],[47,15764,82],{"className":15765},[893],[47,15767],{"className":15768,"style":889},[888],[47,15770,14742],{"className":15771},[508],[47,15773],{"className":15774,"style":1303},[888],[47,15776,15778],{"className":15777},[1338,14600],[47,15779,15781,15816],{"className":15780},[520,521],[47,15782,15784,15813],{"className":15783},[525],[47,15785,15788,15801],{"className":15786,"style":15787},[529],"height:1.05em;",[47,15789,15791,15795],{"style":15790},"top:-1.9em;margin-left:0em;",[47,15792],{"className":15793,"style":15794},[537],"height:3.05em;",[47,15796,15798],{"className":15797},[542,543,544,545],[47,15799,13532],{"className":15800,"style":13548},[508,581,545],[47,15802,15804,15807],{"style":15803},"top:-3.05em;",[47,15805],{"className":15806,"style":15794},[537],[47,15808,15809],{},[47,15810,2213],{"className":15811},[1338,2297,15812],"large-op",[47,15814,616],{"className":15815},[615],[47,15817,15819],{"className":15818},[525],[47,15820,15823],{"className":15821,"style":15822},[529],"height:1.25em;",[47,15824],{},[47,15826],{"className":15827,"style":1303},[888],[47,15829,859],{"className":15830,"style":14471},[508,581],[47,15832,1187],{"className":15833},[512],[47,15835,13532],{"className":15836,"style":13548},[508,581],[47,15838,1201],{"className":15839},[702],[47,15841,15843,15846],{"className":15842},[508],[47,15844,5974],{"className":15845,"style":3555},[508,581],[47,15847,15849],{"className":15848},[585],[47,15850,15852,15872],{"className":15851},[520,521],[47,15853,15855,15869],{"className":15854},[525],[47,15856,15858],{"className":15857,"style":1635},[529],[47,15859,15860,15863],{"style":7830},[47,15861],{"className":15862,"style":929},[537],[47,15864,15866],{"className":15865},[542,543,544,545],[47,15867,14413],{"className":15868,"style":3580},[508,581,545],[47,15870,616],{"className":15871},[615],[47,15873,15875],{"className":15874},[525],[47,15876,15878],{"className":15877,"style":1657},[529],[47,15879],{},[47,15881,1187],{"className":15882},[512],[47,15884,13532],{"className":15885,"style":13548},[508,581],[47,15887,1201],{"className":15888},[702],[47,15890,15891,15894],{"style":15654},[47,15892],{"className":15893,"style":15532},[537],[47,15895,15897,15900,15903,15906,15909,15953,15956,15959,15962,15965,15968,15971,16011,16014,16017],{"className":15896},[508],[47,15898],{"className":15899},[508],[47,15901],{"className":15902,"style":889},[888],[47,15904,82],{"className":15905},[893],[47,15907],{"className":15908,"style":889},[888],[47,15910,15912],{"className":15911},[1338,14600],[47,15913,15915,15945],{"className":15914},[520,521],[47,15916,15918,15942],{"className":15917},[525],[47,15919,15921,15932],{"className":15920,"style":15787},[529],[47,15922,15923,15926],{"style":15790},[47,15924],{"className":15925,"style":15794},[537],[47,15927,15929],{"className":15928},[542,543,544,545],[47,15930,13532],{"className":15931,"style":13548},[508,581,545],[47,15933,15934,15937],{"style":15803},[47,15935],{"className":15936,"style":15794},[537],[47,15938,15939],{},[47,15940,2213],{"className":15941},[1338,2297,15812],[47,15943,616],{"className":15944},[615],[47,15946,15948],{"className":15947},[525],[47,15949,15951],{"className":15950,"style":15822},[529],[47,15952],{},[47,15954],{"className":15955,"style":1303},[888],[47,15957,859],{"className":15958,"style":14471},[508,581],[47,15960,1187],{"className":15961},[512],[47,15963,13532],{"className":15964,"style":13548},[508,581],[47,15966,1201],{"className":15967},[702],[47,15969,14742],{"className":15970},[508],[47,15972,15974,15977],{"className":15973},[508],[47,15975,5974],{"className":15976,"style":3555},[508,581],[47,15978,15980],{"className":15979},[585],[47,15981,15983,16003],{"className":15982},[520,521],[47,15984,15986,16000],{"className":15985},[525],[47,15987,15989],{"className":15988,"style":1635},[529],[47,15990,15991,15994],{"style":7830},[47,15992],{"className":15993,"style":929},[537],[47,15995,15997],{"className":15996},[542,543,544,545],[47,15998,14413],{"className":15999,"style":3580},[508,581,545],[47,16001,616],{"className":16002},[615],[47,16004,16006],{"className":16005},[525],[47,16007,16009],{"className":16008,"style":1657},[529],[47,16010],{},[47,16012,1187],{"className":16013},[512],[47,16015,13532],{"className":16016,"style":13548},[508,581],[47,16018,1201],{"className":16019},[702],[47,16021,16022,16025],{"style":15663},[47,16023],{"className":16024,"style":15532},[537],[47,16026,16028,16031,16034,16037,16040,16084,16087,16090,16093,16096,16099,16102,16142,16145,16148,16151,16154,16157,16160],{"className":16027},[508],[47,16029],{"className":16030},[508],[47,16032],{"className":16033,"style":889},[888],[47,16035,82],{"className":16036},[893],[47,16038],{"className":16039,"style":889},[888],[47,16041,16043],{"className":16042},[1338,14600],[47,16044,16046,16076],{"className":16045},[520,521],[47,16047,16049,16073],{"className":16048},[525],[47,16050,16052,16063],{"className":16051,"style":15787},[529],[47,16053,16054,16057],{"style":15790},[47,16055],{"className":16056,"style":15794},[537],[47,16058,16060],{"className":16059},[542,543,544,545],[47,16061,13532],{"className":16062,"style":13548},[508,581,545],[47,16064,16065,16068],{"style":15803},[47,16066],{"className":16067,"style":15794},[537],[47,16069,16070],{},[47,16071,2213],{"className":16072},[1338,2297,15812],[47,16074,616],{"className":16075},[615],[47,16077,16079],{"className":16078},[525],[47,16080,16082],{"className":16081,"style":15822},[529],[47,16083],{},[47,16085],{"className":16086,"style":1303},[888],[47,16088,859],{"className":16089,"style":14471},[508,581],[47,16091,1187],{"className":16092},[512],[47,16094,13532],{"className":16095,"style":13548},[508,581],[47,16097,1201],{"className":16098},[702],[47,16100,14742],{"className":16101},[508],[47,16103,16105,16108],{"className":16104},[508],[47,16106,5974],{"className":16107,"style":3555},[508,581],[47,16109,16111],{"className":16110},[585],[47,16112,16114,16134],{"className":16113},[520,521],[47,16115,16117,16131],{"className":16116},[525],[47,16118,16120],{"className":16119,"style":1635},[529],[47,16121,16122,16125],{"style":7830},[47,16123],{"className":16124,"style":929},[537],[47,16126,16128],{"className":16127},[542,543,544,545],[47,16129,14413],{"className":16130,"style":3580},[508,581,545],[47,16132,616],{"className":16133},[615],[47,16135,16137],{"className":16136},[525],[47,16138,16140],{"className":16139,"style":1657},[529],[47,16141],{},[47,16143,1187],{"className":16144},[512],[47,16146,13532],{"className":16147,"style":13548},[508,581],[47,16149,1201],{"className":16150},[702],[47,16152],{"className":16153,"style":2259},[888],[47,16155,2203],{"className":16156},[942],[47,16158],{"className":16159,"style":2259},[888],[47,16161,16163,16166,16315],{"className":16162},[508],[47,16164],{"className":16165},[512,513],[47,16167,16169],{"className":16168},[467],[47,16170,16172,16306],{"className":16171},[520,521],[47,16173,16175,16303],{"className":16174},[525],[47,16176,16179,16237,16245],{"className":16177,"style":16178},[529],"height:1.427em;",[47,16180,16182,16185],{"style":16181},"top:-2.314em;",[47,16183],{"className":16184,"style":538},[537],[47,16186,16188,16228,16231,16234],{"className":16187},[508],[47,16189,16191,16194],{"className":16190},[508],[47,16192,5974],{"className":16193,"style":3555},[508,581],[47,16195,16197],{"className":16196},[585],[47,16198,16200,16220],{"className":16199},[520,521],[47,16201,16203,16217],{"className":16202},[525],[47,16204,16206],{"className":16205,"style":1635},[529],[47,16207,16208,16211],{"style":7830},[47,16209],{"className":16210,"style":929},[537],[47,16212,16214],{"className":16213},[542,543,544,545],[47,16215,14413],{"className":16216,"style":3580},[508,581,545],[47,16218,616],{"className":16219},[615],[47,16221,16223],{"className":16222},[525],[47,16224,16226],{"className":16225,"style":1657},[529],[47,16227],{},[47,16229,1187],{"className":16230},[512],[47,16232,13532],{"className":16233,"style":13548},[508,581],[47,16235,1201],{"className":16236},[702],[47,16238,16239,16242],{"style":663},[47,16240],{"className":16241,"style":538},[537],[47,16243],{"className":16244,"style":671},[670],[47,16246,16248,16251],{"style":16247},"top:-3.677em;",[47,16249],{"className":16250,"style":538},[537],[47,16252,16254,16294,16297,16300],{"className":16253},[508],[47,16255,16257,16260],{"className":16256},[508],[47,16258,5974],{"className":16259,"style":3555},[508,581],[47,16261,16263],{"className":16262},[585],[47,16264,16266,16286],{"className":16265},[520,521],[47,16267,16269,16283],{"className":16268},[525],[47,16270,16272],{"className":16271,"style":1635},[529],[47,16273,16274,16277],{"style":7830},[47,16275],{"className":16276,"style":929},[537],[47,16278,16280],{"className":16279},[542,543,544,545],[47,16281,14413],{"className":16282,"style":3580},[508,581,545],[47,16284,616],{"className":16285},[615],[47,16287,16289],{"className":16288},[525],[47,16290,16292],{"className":16291,"style":1657},[529],[47,16293],{},[47,16295,1187],{"className":16296},[512],[47,16298,13532],{"className":16299,"style":13548},[508,581],[47,16301,1201],{"className":16302},[702],[47,16304,616],{"className":16305},[615],[47,16307,16309],{"className":16308},[525],[47,16310,16313],{"className":16311,"style":16312},[529],"height:0.936em;",[47,16314],{},[47,16316],{"className":16317},[702,513],[47,16319,16320,16323],{"style":15672},[47,16321],{"className":16322,"style":15532},[537],[47,16324,16326,16329,16332,16335,16338,16382,16385,16425,16428,16431,16434,16437,16440,16443,16446],{"className":16325},[508],[47,16327],{"className":16328},[508],[47,16330],{"className":16331,"style":889},[888],[47,16333,82],{"className":16334},[893],[47,16336],{"className":16337,"style":889},[888],[47,16339,16341],{"className":16340},[1338,14600],[47,16342,16344,16374],{"className":16343},[520,521],[47,16345,16347,16371],{"className":16346},[525],[47,16348,16350,16361],{"className":16349,"style":15787},[529],[47,16351,16352,16355],{"style":15790},[47,16353],{"className":16354,"style":15794},[537],[47,16356,16358],{"className":16357},[542,543,544,545],[47,16359,13532],{"className":16360,"style":13548},[508,581,545],[47,16362,16363,16366],{"style":15803},[47,16364],{"className":16365,"style":15794},[537],[47,16367,16368],{},[47,16369,2213],{"className":16370},[1338,2297,15812],[47,16372,616],{"className":16373},[615],[47,16375,16377],{"className":16376},[525],[47,16378,16380],{"className":16379,"style":15822},[529],[47,16381],{},[47,16383],{"className":16384,"style":1303},[888],[47,16386,16388,16391],{"className":16387},[508],[47,16389,5974],{"className":16390,"style":3555},[508,581],[47,16392,16394],{"className":16393},[585],[47,16395,16397,16417],{"className":16396},[520,521],[47,16398,16400,16414],{"className":16399},[525],[47,16401,16403],{"className":16402,"style":1635},[529],[47,16404,16405,16408],{"style":7830},[47,16406],{"className":16407,"style":929},[537],[47,16409,16411],{"className":16410},[542,543,544,545],[47,16412,14413],{"className":16413,"style":3580},[508,581,545],[47,16415,616],{"className":16416},[615],[47,16418,16420],{"className":16419},[525],[47,16421,16423],{"className":16422,"style":1657},[529],[47,16424],{},[47,16426,1187],{"className":16427},[512],[47,16429,13532],{"className":16430,"style":13548},[508,581],[47,16432,1201],{"className":16433},[702],[47,16435,859],{"className":16436,"style":14471},[508,581],[47,16438,1187],{"className":16439},[512],[47,16441,13532],{"className":16442,"style":13548},[508,581],[47,16444,1201],{"className":16445},[702],[47,16447,16449,16452,16600],{"className":16448},[508],[47,16450],{"className":16451},[512,513],[47,16453,16455],{"className":16454},[467],[47,16456,16458,16592],{"className":16457},[520,521],[47,16459,16461,16589],{"className":16460},[525],[47,16462,16464,16521,16529],{"className":16463,"style":16178},[529],[47,16465,16466,16469],{"style":16181},[47,16467],{"className":16468,"style":538},[537],[47,16470,16472,16512,16515,16518],{"className":16471},[508],[47,16473,16475,16478],{"className":16474},[508],[47,16476,5974],{"className":16477,"style":3555},[508,581],[47,16479,16481],{"className":16480},[585],[47,16482,16484,16504],{"className":16483},[520,521],[47,16485,16487,16501],{"className":16486},[525],[47,16488,16490],{"className":16489,"style":1635},[529],[47,16491,16492,16495],{"style":7830},[47,16493],{"className":16494,"style":929},[537],[47,16496,16498],{"className":16497},[542,543,544,545],[47,16499,14413],{"className":16500,"style":3580},[508,581,545],[47,16502,616],{"className":16503},[615],[47,16505,16507],{"className":16506},[525],[47,16508,16510],{"className":16509,"style":1657},[529],[47,16511],{},[47,16513,1187],{"className":16514},[512],[47,16516,13532],{"className":16517,"style":13548},[508,581],[47,16519,1201],{"className":16520},[702],[47,16522,16523,16526],{"style":663},[47,16524],{"className":16525,"style":538},[537],[47,16527],{"className":16528,"style":671},[670],[47,16530,16531,16534],{"style":16247},[47,16532],{"className":16533,"style":538},[537],[47,16535,16537,16540,16580,16583,16586],{"className":16536},[508],[47,16538,14742],{"className":16539},[508],[47,16541,16543,16546],{"className":16542},[508],[47,16544,5974],{"className":16545,"style":3555},[508,581],[47,16547,16549],{"className":16548},[585],[47,16550,16552,16572],{"className":16551},[520,521],[47,16553,16555,16569],{"className":16554},[525],[47,16556,16558],{"className":16557,"style":1635},[529],[47,16559,16560,16563],{"style":7830},[47,16561],{"className":16562,"style":929},[537],[47,16564,16566],{"className":16565},[542,543,544,545],[47,16567,14413],{"className":16568,"style":3580},[508,581,545],[47,16570,616],{"className":16571},[615],[47,16573,16575],{"className":16574},[525],[47,16576,16578],{"className":16577,"style":1657},[529],[47,16579],{},[47,16581,1187],{"className":16582},[512],[47,16584,13532],{"className":16585,"style":13548},[508,581],[47,16587,1201],{"className":16588},[702],[47,16590,616],{"className":16591},[615],[47,16593,16595],{"className":16594},[525],[47,16596,16598],{"className":16597,"style":16312},[529],[47,16599],{},[47,16601],{"className":16602},[702,513],[47,16604,16605,16608],{"style":15681},[47,16606],{"className":16607,"style":15532},[537],[47,16609,16611,16614,16617,16620,16623,16687,16690,16758,16761,16764,16767,16798,16801],{"className":16610},[508],[47,16612],{"className":16613},[508],[47,16615],{"className":16616,"style":889},[888],[47,16618,15037],{"className":16619},[893],[47,16621],{"className":16622,"style":889},[888],[47,16624,16626,16629,16684],{"className":16625},[508],[47,16627],{"className":16628},[512,513],[47,16630,16632],{"className":16631},[467],[47,16633,16635,16675],{"className":16634},[520,521],[47,16636,16638,16672],{"className":16637},[525],[47,16639,16642,16653,16661],{"className":16640,"style":16641},[529],"height:1.3214em;",[47,16643,16644,16647],{"style":16181},[47,16645],{"className":16646,"style":538},[537],[47,16648,16650],{"className":16649},[508],[47,16651,7692],{"className":16652,"style":7767},[508,581],[47,16654,16655,16658],{"style":663},[47,16656],{"className":16657,"style":538},[537],[47,16659],{"className":16660,"style":671},[670],[47,16662,16663,16666],{"style":16247},[47,16664],{"className":16665,"style":538},[537],[47,16667,16669],{"className":16668},[508],[47,16670,472],{"className":16671},[508],[47,16673,616],{"className":16674},[615],[47,16676,16678],{"className":16677},[525],[47,16679,16682],{"className":16680,"style":16681},[529],"height:0.686em;",[47,16683],{},[47,16685],{"className":16686},[702,513],[47,16688],{"className":16689,"style":1303},[888],[47,16691,16693],{"className":16692},[1338,14600],[47,16694,16696,16749],{"className":16695},[520,521],[47,16697,16699,16746],{"className":16698},[525],[47,16700,16703,16724,16734],{"className":16701,"style":16702},[529],"height:1.8283em;",[47,16704,16706,16709],{"style":16705},"top:-1.8829em;margin-left:0em;",[47,16707],{"className":16708,"style":15794},[537],[47,16710,16712],{"className":16711},[542,543,544,545],[47,16713,16715,16718,16721],{"className":16714},[508,545],[47,16716,724],{"className":16717},[508,581,545],[47,16719,82],{"className":16720},[893,545],[47,16722,472],{"className":16723},[508,545],[47,16725,16726,16729],{"style":15803},[47,16727],{"className":16728,"style":15794},[537],[47,16730,16731],{},[47,16732,2213],{"className":16733},[1338,2297,15812],[47,16735,16737,16740],{"style":16736},"top:-4.3em;margin-left:0em;",[47,16738],{"className":16739,"style":15794},[537],[47,16741,16743],{"className":16742},[542,543,544,545],[47,16744,7692],{"className":16745,"style":7767},[508,581,545],[47,16747,616],{"className":16748},[615],[47,16750,16752],{"className":16751},[525],[47,16753,16756],{"className":16754,"style":16755},[529],"height:1.2671em;",[47,16757],{},[47,16759],{"className":16760,"style":1303},[888],[47,16762,859],{"className":16763,"style":14471},[508,581],[47,16765,1187],{"className":16766},[512],[47,16768,16770,16773],{"className":16769},[508],[47,16771,13532],{"className":16772,"style":13548},[508,581],[47,16774,16776],{"className":16775},[585],[47,16777,16779],{"className":16778},[520],[47,16780,16782],{"className":16781},[525],[47,16783,16786],{"className":16784,"style":16785},[529],"height:0.7144em;",[47,16787,16789,16792],{"style":16788},"top:-3.113em;margin-right:0.05em;",[47,16790],{"className":16791,"style":929},[537],[47,16793,16795],{"className":16794},[542,543,544,545],[47,16796,724],{"className":16797},[508,581,545],[47,16799,1201],{"className":16800},[702],[47,16802,16804,16807,17010],{"className":16803},[508],[47,16805],{"className":16806},[512,513],[47,16808,16810],{"className":16809},[467],[47,16811,16813,17002],{"className":16812},[520,521],[47,16814,16816,16999],{"className":16815},[525],[47,16817,16819,16904,16912],{"className":16818,"style":16178},[529],[47,16820,16821,16824],{"style":16181},[47,16822],{"className":16823,"style":538},[537],[47,16825,16827,16867,16870,16901],{"className":16826},[508],[47,16828,16830,16833],{"className":16829},[508],[47,16831,5974],{"className":16832,"style":3555},[508,581],[47,16834,16836],{"className":16835},[585],[47,16837,16839,16859],{"className":16838},[520,521],[47,16840,16842,16856],{"className":16841},[525],[47,16843,16845],{"className":16844,"style":1635},[529],[47,16846,16847,16850],{"style":7830},[47,16848],{"className":16849,"style":929},[537],[47,16851,16853],{"className":16852},[542,543,544,545],[47,16854,14413],{"className":16855,"style":3580},[508,581,545],[47,16857,616],{"className":16858},[615],[47,16860,16862],{"className":16861},[525],[47,16863,16865],{"className":16864,"style":1657},[529],[47,16866],{},[47,16868,1187],{"className":16869},[512],[47,16871,16873,16876],{"className":16872},[508],[47,16874,13532],{"className":16875,"style":13548},[508,581],[47,16877,16879],{"className":16878},[585],[47,16880,16882],{"className":16881},[520],[47,16883,16885],{"className":16884},[525],[47,16886,16889],{"className":16887,"style":16888},[529],"height:0.5904em;",[47,16890,16892,16895],{"style":16891},"top:-2.989em;margin-right:0.05em;",[47,16893],{"className":16894,"style":929},[537],[47,16896,16898],{"className":16897},[542,543,544,545],[47,16899,724],{"className":16900},[508,581,545],[47,16902,1201],{"className":16903},[702],[47,16905,16906,16909],{"style":663},[47,16907],{"className":16908,"style":538},[537],[47,16910],{"className":16911,"style":671},[670],[47,16913,16914,16917],{"style":16247},[47,16915],{"className":16916,"style":538},[537],[47,16918,16920,16923,16963,16966,16996],{"className":16919},[508],[47,16921,14742],{"className":16922},[508],[47,16924,16926,16929],{"className":16925},[508],[47,16927,14],{"className":16928},[508,581],[47,16930,16932],{"className":16931},[585],[47,16933,16935,16955],{"className":16934},[520,521],[47,16936,16938,16952],{"className":16937},[525],[47,16939,16941],{"className":16940,"style":1635},[529],[47,16942,16943,16946],{"style":1638},[47,16944],{"className":16945,"style":929},[537],[47,16947,16949],{"className":16948},[542,543,544,545],[47,16950,14413],{"className":16951,"style":3580},[508,581,545],[47,16953,616],{"className":16954},[615],[47,16956,16958],{"className":16957},[525],[47,16959,16961],{"className":16960,"style":1657},[529],[47,16962],{},[47,16964,1187],{"className":16965},[512],[47,16967,16969,16972],{"className":16968},[508],[47,16970,13532],{"className":16971,"style":13548},[508,581],[47,16973,16975],{"className":16974},[585],[47,16976,16978],{"className":16977},[520],[47,16979,16981],{"className":16980},[525],[47,16982,16985],{"className":16983,"style":16984},[529],"height:0.6644em;",[47,16986,16987,16990],{"style":925},[47,16988],{"className":16989,"style":929},[537],[47,16991,16993],{"className":16992},[542,543,544,545],[47,16994,724],{"className":16995},[508,581,545],[47,16997,1201],{"className":16998},[702],[47,17000,616],{"className":17001},[615],[47,17003,17005],{"className":17004},[525],[47,17006,17008],{"className":17007,"style":16312},[529],[47,17009],{},[47,17011],{"className":17012},[702,513],[47,17014,17015,17018],{"style":15690},[47,17016],{"className":17017,"style":15532},[537],[47,17019,17021,17024,17027,17030,17033,17095,17098,17162,17165,17168,17171,17200,17203,17206,17209,17217,17220,17260,17263,17292],{"className":17020},[508],[47,17022],{"className":17023},[508],[47,17025],{"className":17026,"style":889},[888],[47,17028,82],{"className":17029},[893],[47,17031],{"className":17032,"style":889},[888],[47,17034,17036,17039,17092],{"className":17035},[508],[47,17037],{"className":17038},[512,513],[47,17040,17042],{"className":17041},[467],[47,17043,17045,17084],{"className":17044},[520,521],[47,17046,17048,17081],{"className":17047},[525],[47,17049,17051,17062,17070],{"className":17050,"style":16641},[529],[47,17052,17053,17056],{"style":16181},[47,17054],{"className":17055,"style":538},[537],[47,17057,17059],{"className":17058},[508],[47,17060,7692],{"className":17061,"style":7767},[508,581],[47,17063,17064,17067],{"style":663},[47,17065],{"className":17066,"style":538},[537],[47,17068],{"className":17069,"style":671},[670],[47,17071,17072,17075],{"style":16247},[47,17073],{"className":17074,"style":538},[537],[47,17076,17078],{"className":17077},[508],[47,17079,472],{"className":17080},[508],[47,17082,616],{"className":17083},[615],[47,17085,17087],{"className":17086},[525],[47,17088,17090],{"className":17089,"style":16681},[529],[47,17091],{},[47,17093],{"className":17094},[702,513],[47,17096],{"className":17097,"style":1303},[888],[47,17099,17101],{"className":17100},[1338,14600],[47,17102,17104,17154],{"className":17103},[520,521],[47,17105,17107,17151],{"className":17106},[525],[47,17108,17110,17130,17140],{"className":17109,"style":16702},[529],[47,17111,17112,17115],{"style":16705},[47,17113],{"className":17114,"style":15794},[537],[47,17116,17118],{"className":17117},[542,543,544,545],[47,17119,17121,17124,17127],{"className":17120},[508,545],[47,17122,724],{"className":17123},[508,581,545],[47,17125,82],{"className":17126},[893,545],[47,17128,472],{"className":17129},[508,545],[47,17131,17132,17135],{"style":15803},[47,17133],{"className":17134,"style":15794},[537],[47,17136,17137],{},[47,17138,2213],{"className":17139},[1338,2297,15812],[47,17141,17142,17145],{"style":16736},[47,17143],{"className":17144,"style":15794},[537],[47,17146,17148],{"className":17147},[542,543,544,545],[47,17149,7692],{"className":17150,"style":7767},[508,581,545],[47,17152,616],{"className":17153},[615],[47,17155,17157],{"className":17156},[525],[47,17158,17160],{"className":17159,"style":16755},[529],[47,17161],{},[47,17163],{"className":17164,"style":1303},[888],[47,17166,859],{"className":17167,"style":14471},[508,581],[47,17169,1187],{"className":17170},[512],[47,17172,17174,17177],{"className":17173},[508],[47,17175,13532],{"className":17176,"style":13548},[508,581],[47,17178,17180],{"className":17179},[585],[47,17181,17183],{"className":17182},[520],[47,17184,17186],{"className":17185},[525],[47,17187,17189],{"className":17188,"style":16785},[529],[47,17190,17191,17194],{"style":16788},[47,17192],{"className":17193,"style":929},[537],[47,17195,17197],{"className":17196},[542,543,544,545],[47,17198,724],{"className":17199},[508,581,545],[47,17201,1201],{"className":17202},[702],[47,17204,14742],{"className":17205},[508],[47,17207],{"className":17208,"style":1303},[888],[47,17210,17212,17213],{"className":17211},[1338],"lo",[47,17214,17216],{"style":17215},"margin-right:0.01389em;","g",[47,17218],{"className":17219,"style":1303},[888],[47,17221,17223,17226],{"className":17222},[508],[47,17224,5974],{"className":17225,"style":3555},[508,581],[47,17227,17229],{"className":17228},[585],[47,17230,17232,17252],{"className":17231},[520,521],[47,17233,17235,17249],{"className":17234},[525],[47,17236,17238],{"className":17237,"style":1635},[529],[47,17239,17240,17243],{"style":7830},[47,17241],{"className":17242,"style":929},[537],[47,17244,17246],{"className":17245},[542,543,544,545],[47,17247,14413],{"className":17248,"style":3580},[508,581,545],[47,17250,616],{"className":17251},[615],[47,17253,17255],{"className":17254},[525],[47,17256,17258],{"className":17257,"style":1657},[529],[47,17259],{},[47,17261,1187],{"className":17262},[512],[47,17264,17266,17269],{"className":17265},[508],[47,17267,13532],{"className":17268,"style":13548},[508,581],[47,17270,17272],{"className":17271},[585],[47,17273,17275],{"className":17274},[520],[47,17276,17278],{"className":17277},[525],[47,17279,17281],{"className":17280,"style":16785},[529],[47,17282,17283,17286],{"style":16788},[47,17284],{"className":17285,"style":929},[537],[47,17287,17289],{"className":17288},[542,543,544,545],[47,17290,724],{"className":17291},[508,581,545],[47,17293,1201],{"className":17294},[702],[47,17296,17297,17300],{"style":15699},[47,17298],{"className":17299,"style":15532},[537],[47,17301,17303,17306,17309,17312,17315,17377,17380,17444,17447,17450,17453,17482,17485,17488,17491,17496,17499,17606,17609,17649,17652,17706,17709,17760],{"className":17302},[508],[47,17304],{"className":17305},[508],[47,17307],{"className":17308,"style":889},[888],[47,17310,82],{"className":17311},[893],[47,17313],{"className":17314,"style":889},[888],[47,17316,17318,17321,17374],{"className":17317},[508],[47,17319],{"className":17320},[512,513],[47,17322,17324],{"className":17323},[467],[47,17325,17327,17366],{"className":17326},[520,521],[47,17328,17330,17363],{"className":17329},[525],[47,17331,17333,17344,17352],{"className":17332,"style":16641},[529],[47,17334,17335,17338],{"style":16181},[47,17336],{"className":17337,"style":538},[537],[47,17339,17341],{"className":17340},[508],[47,17342,7692],{"className":17343,"style":7767},[508,581],[47,17345,17346,17349],{"style":663},[47,17347],{"className":17348,"style":538},[537],[47,17350],{"className":17351,"style":671},[670],[47,17353,17354,17357],{"style":16247},[47,17355],{"className":17356,"style":538},[537],[47,17358,17360],{"className":17359},[508],[47,17361,472],{"className":17362},[508],[47,17364,616],{"className":17365},[615],[47,17367,17369],{"className":17368},[525],[47,17370,17372],{"className":17371,"style":16681},[529],[47,17373],{},[47,17375],{"className":17376},[702,513],[47,17378],{"className":17379,"style":1303},[888],[47,17381,17383],{"className":17382},[1338,14600],[47,17384,17386,17436],{"className":17385},[520,521],[47,17387,17389,17433],{"className":17388},[525],[47,17390,17392,17412,17422],{"className":17391,"style":16702},[529],[47,17393,17394,17397],{"style":16705},[47,17395],{"className":17396,"style":15794},[537],[47,17398,17400],{"className":17399},[542,543,544,545],[47,17401,17403,17406,17409],{"className":17402},[508,545],[47,17404,724],{"className":17405},[508,581,545],[47,17407,82],{"className":17408},[893,545],[47,17410,472],{"className":17411},[508,545],[47,17413,17414,17417],{"style":15803},[47,17415],{"className":17416,"style":15794},[537],[47,17418,17419],{},[47,17420,2213],{"className":17421},[1338,2297,15812],[47,17423,17424,17427],{"style":16736},[47,17425],{"className":17426,"style":15794},[537],[47,17428,17430],{"className":17429},[542,543,544,545],[47,17431,7692],{"className":17432,"style":7767},[508,581,545],[47,17434,616],{"className":17435},[615],[47,17437,17439],{"className":17438},[525],[47,17440,17442],{"className":17441,"style":16755},[529],[47,17443],{},[47,17445],{"className":17446,"style":1303},[888],[47,17448,859],{"className":17449,"style":14471},[508,581],[47,17451,1187],{"className":17452},[512],[47,17454,17456,17459],{"className":17455},[508],[47,17457,13532],{"className":17458,"style":13548},[508,581],[47,17460,17462],{"className":17461},[585],[47,17463,17465],{"className":17464},[520],[47,17466,17468],{"className":17467},[525],[47,17469,17471],{"className":17470,"style":16785},[529],[47,17472,17473,17476],{"style":16788},[47,17474],{"className":17475,"style":929},[537],[47,17477,17479],{"className":17478},[542,543,544,545],[47,17480,724],{"className":17481},[508,581,545],[47,17483,1201],{"className":17484},[702],[47,17486,14742],{"className":17487},[508],[47,17489],{"className":17490,"style":1303},[888],[47,17492,17212,17494],{"className":17493},[1338],[47,17495,17216],{"style":17215},[47,17497],{"className":17498,"style":1303},[888],[47,17500,17502],{"className":17501},[1338,14600],[47,17503,17505,17598],{"className":17504},[520,521],[47,17506,17508,17595],{"className":17507},[525],[47,17509,17512,17532,17542],{"className":17510,"style":17511},[529],"height:1.8394em;",[47,17513,17514,17517],{"style":16705},[47,17515],{"className":17516,"style":15794},[537],[47,17518,17520],{"className":17519},[542,543,544,545],[47,17521,17523,17526,17529],{"className":17522},[508,545],[47,17524,1166],{"className":17525},[508,581,545],[47,17527,82],{"className":17528},[893,545],[47,17530,472],{"className":17531},[508,545],[47,17533,17534,17537],{"style":15803},[47,17535],{"className":17536,"style":15794},[537],[47,17538,17539],{},[47,17540,15246],{"className":17541},[1338,2297,15812],[47,17543,17545,17548],{"style":17544},"top:-4.3111em;margin-left:0em;",[47,17546],{"className":17547,"style":15794},[537],[47,17549,17551],{"className":17550},[542,543,544,545],[47,17552,17554],{"className":17553},[508,545],[47,17555,17557,17560],{"className":17556},[508,545],[47,17558,13604],{"className":17559,"style":3555},[508,581,545],[47,17561,17563],{"className":17562},[585],[47,17564,17566,17587],{"className":17565},[520,521],[47,17567,17569,17584],{"className":17568},[525],[47,17570,17572],{"className":17571,"style":4672},[529],[47,17573,17575,17578],{"style":17574},"top:-2.357em;margin-left:-0.1389em;margin-right:0.0714em;",[47,17576],{"className":17577,"style":602},[537],[47,17579,17581],{"className":17580},[542,606,607,545],[47,17582,724],{"className":17583},[508,581,545],[47,17585,616],{"className":17586},[615],[47,17588,17590],{"className":17589},[525],[47,17591,17593],{"className":17592,"style":4694},[529],[47,17594],{},[47,17596,616],{"className":17597},[615],[47,17599,17601],{"className":17600},[525],[47,17602,17604],{"className":17603,"style":16755},[529],[47,17605],{},[47,17607],{"className":17608,"style":1303},[888],[47,17610,17612,17615],{"className":17611},[508],[47,17613,5974],{"className":17614,"style":3555},[508,581],[47,17616,17618],{"className":17617},[585],[47,17619,17621,17641],{"className":17620},[520,521],[47,17622,17624,17638],{"className":17623},[525],[47,17625,17627],{"className":17626,"style":1635},[529],[47,17628,17629,17632],{"style":7830},[47,17630],{"className":17631,"style":929},[537],[47,17633,17635],{"className":17634},[542,543,544,545],[47,17636,14413],{"className":17637,"style":3580},[508,581,545],[47,17639,616],{"className":17640},[615],[47,17642,17644],{"className":17643},[525],[47,17645,17647],{"className":17646,"style":1657},[529],[47,17648],{},[47,17650,1187],{"className":17651},[512],[47,17653,17655,17658],{"className":17654},[508],[47,17656,285],{"className":17657},[508,581],[47,17659,17661],{"className":17660},[585],[47,17662,17664,17697],{"className":17663},[520,521],[47,17665,17667,17694],{"className":17666},[525],[47,17668,17671,17683],{"className":17669,"style":17670},[529],"height:0.8436em;",[47,17672,17674,17677],{"style":17673},"top:-2.453em;margin-left:0em;margin-right:0.05em;",[47,17675],{"className":17676,"style":929},[537],[47,17678,17680],{"className":17679},[542,543,544,545],[47,17681,724],{"className":17682},[508,581,545],[47,17684,17685,17688],{"style":16788},[47,17686],{"className":17687,"style":929},[537],[47,17689,17691],{"className":17690},[542,543,544,545],[47,17692,1166],{"className":17693},[508,581,545],[47,17695,616],{"className":17696},[615],[47,17698,17700],{"className":17699},[525],[47,17701,17704],{"className":17702,"style":17703},[529],"height:0.247em;",[47,17705],{},[47,17707,13303],{"className":17708},[508],[47,17710,17712,17715],{"className":17711},[508],[47,17713,13164],{"className":17714},[508,581],[47,17716,17718],{"className":17717},[585],[47,17719,17721,17752],{"className":17720},[520,521],[47,17722,17724,17749],{"className":17723},[525],[47,17725,17727,17738],{"className":17726,"style":17670},[529],[47,17728,17729,17732],{"style":17673},[47,17730],{"className":17731,"style":929},[537],[47,17733,17735],{"className":17734},[542,543,544,545],[47,17736,724],{"className":17737},[508,581,545],[47,17739,17740,17743],{"style":16788},[47,17741],{"className":17742,"style":929},[537],[47,17744,17746],{"className":17745},[542,543,544,545],[47,17747,1166],{"className":17748},[508,581,545],[47,17750,616],{"className":17751},[615],[47,17753,17755],{"className":17754},[525],[47,17756,17758],{"className":17757,"style":17703},[529],[47,17759],{},[47,17761,1201],{"className":17762},[702],[47,17764,17765,17768],{"style":15708},[47,17766],{"className":17767,"style":15532},[537],[47,17769,17771,17774,17777,17780,17783,17845,17848,17912,17915,17918,17921,17950,17953,17956,18060,18063,18066,18069,18074,18077,18117,18120,18171,18174,18225],{"className":17770},[508],[47,17772],{"className":17773},[508],[47,17775],{"className":17776,"style":889},[888],[47,17778,82],{"className":17779},[893],[47,17781],{"className":17782,"style":889},[888],[47,17784,17786,17789,17842],{"className":17785},[508],[47,17787],{"className":17788},[512,513],[47,17790,17792],{"className":17791},[467],[47,17793,17795,17834],{"className":17794},[520,521],[47,17796,17798,17831],{"className":17797},[525],[47,17799,17801,17812,17820],{"className":17800,"style":16641},[529],[47,17802,17803,17806],{"style":16181},[47,17804],{"className":17805,"style":538},[537],[47,17807,17809],{"className":17808},[508],[47,17810,7692],{"className":17811,"style":7767},[508,581],[47,17813,17814,17817],{"style":663},[47,17815],{"className":17816,"style":538},[537],[47,17818],{"className":17819,"style":671},[670],[47,17821,17822,17825],{"style":16247},[47,17823],{"className":17824,"style":538},[537],[47,17826,17828],{"className":17827},[508],[47,17829,472],{"className":17830},[508],[47,17832,616],{"className":17833},[615],[47,17835,17837],{"className":17836},[525],[47,17838,17840],{"className":17839,"style":16681},[529],[47,17841],{},[47,17843],{"className":17844},[702,513],[47,17846],{"className":17847,"style":1303},[888],[47,17849,17851],{"className":17850},[1338,14600],[47,17852,17854,17904],{"className":17853},[520,521],[47,17855,17857,17901],{"className":17856},[525],[47,17858,17860,17880,17890],{"className":17859,"style":16702},[529],[47,17861,17862,17865],{"style":16705},[47,17863],{"className":17864,"style":15794},[537],[47,17866,17868],{"className":17867},[542,543,544,545],[47,17869,17871,17874,17877],{"className":17870},[508,545],[47,17872,724],{"className":17873},[508,581,545],[47,17875,82],{"className":17876},[893,545],[47,17878,472],{"className":17879},[508,545],[47,17881,17882,17885],{"style":15803},[47,17883],{"className":17884,"style":15794},[537],[47,17886,17887],{},[47,17888,2213],{"className":17889},[1338,2297,15812],[47,17891,17892,17895],{"style":16736},[47,17893],{"className":17894,"style":15794},[537],[47,17896,17898],{"className":17897},[542,543,544,545],[47,17899,7692],{"className":17900,"style":7767},[508,581,545],[47,17902,616],{"className":17903},[615],[47,17905,17907],{"className":17906},[525],[47,17908,17910],{"className":17909,"style":16755},[529],[47,17911],{},[47,17913],{"className":17914,"style":1303},[888],[47,17916,859],{"className":17917,"style":14471},[508,581],[47,17919,1187],{"className":17920},[512],[47,17922,17924,17927],{"className":17923},[508],[47,17925,13532],{"className":17926,"style":13548},[508,581],[47,17928,17930],{"className":17929},[585],[47,17931,17933],{"className":17932},[520],[47,17934,17936],{"className":17935},[525],[47,17937,17939],{"className":17938,"style":16785},[529],[47,17940,17941,17944],{"style":16788},[47,17942],{"className":17943,"style":929},[537],[47,17945,17947],{"className":17946},[542,543,544,545],[47,17948,724],{"className":17949},[508,581,545],[47,17951,1201],{"className":17952},[702],[47,17954],{"className":17955,"style":1303},[888],[47,17957,17959],{"className":17958},[1338,14600],[47,17960,17962,18052],{"className":17961},[520,521],[47,17963,17965,18049],{"className":17964},[525],[47,17966,17968,17988,17998],{"className":17967,"style":17511},[529],[47,17969,17970,17973],{"style":16705},[47,17971],{"className":17972,"style":15794},[537],[47,17974,17976],{"className":17975},[542,543,544,545],[47,17977,17979,17982,17985],{"className":17978},[508,545],[47,17980,1166],{"className":17981},[508,581,545],[47,17983,82],{"className":17984},[893,545],[47,17986,472],{"className":17987},[508,545],[47,17989,17990,17993],{"style":15803},[47,17991],{"className":17992,"style":15794},[537],[47,17994,17995],{},[47,17996,2213],{"className":17997},[1338,2297,15812],[47,17999,18000,18003],{"style":17544},[47,18001],{"className":18002,"style":15794},[537],[47,18004,18006],{"className":18005},[542,543,544,545],[47,18007,18009],{"className":18008},[508,545],[47,18010,18012,18015],{"className":18011},[508,545],[47,18013,13604],{"className":18014,"style":3555},[508,581,545],[47,18016,18018],{"className":18017},[585],[47,18019,18021,18041],{"className":18020},[520,521],[47,18022,18024,18038],{"className":18023},[525],[47,18025,18027],{"className":18026,"style":4672},[529],[47,18028,18029,18032],{"style":17574},[47,18030],{"className":18031,"style":602},[537],[47,18033,18035],{"className":18034},[542,606,607,545],[47,18036,724],{"className":18037},[508,581,545],[47,18039,616],{"className":18040},[615],[47,18042,18044],{"className":18043},[525],[47,18045,18047],{"className":18046,"style":4694},[529],[47,18048],{},[47,18050,616],{"className":18051},[615],[47,18053,18055],{"className":18054},[525],[47,18056,18058],{"className":18057,"style":16755},[529],[47,18059],{},[47,18061],{"className":18062,"style":1303},[888],[47,18064,14742],{"className":18065},[508],[47,18067],{"className":18068,"style":1303},[888],[47,18070,17212,18072],{"className":18071},[1338],[47,18073,17216],{"style":17215},[47,18075],{"className":18076,"style":1303},[888],[47,18078,18080,18083],{"className":18079},[508],[47,18081,5974],{"className":18082,"style":3555},[508,581],[47,18084,18086],{"className":18085},[585],[47,18087,18089,18109],{"className":18088},[520,521],[47,18090,18092,18106],{"className":18091},[525],[47,18093,18095],{"className":18094,"style":1635},[529],[47,18096,18097,18100],{"style":7830},[47,18098],{"className":18099,"style":929},[537],[47,18101,18103],{"className":18102},[542,543,544,545],[47,18104,14413],{"className":18105,"style":3580},[508,581,545],[47,18107,616],{"className":18108},[615],[47,18110,18112],{"className":18111},[525],[47,18113,18115],{"className":18114,"style":1657},[529],[47,18116],{},[47,18118,1187],{"className":18119},[512],[47,18121,18123,18126],{"className":18122},[508],[47,18124,285],{"className":18125},[508,581],[47,18127,18129],{"className":18128},[585],[47,18130,18132,18163],{"className":18131},[520,521],[47,18133,18135,18160],{"className":18134},[525],[47,18136,18138,18149],{"className":18137,"style":17670},[529],[47,18139,18140,18143],{"style":17673},[47,18141],{"className":18142,"style":929},[537],[47,18144,18146],{"className":18145},[542,543,544,545],[47,18147,724],{"className":18148},[508,581,545],[47,18150,18151,18154],{"style":16788},[47,18152],{"className":18153,"style":929},[537],[47,18155,18157],{"className":18156},[542,543,544,545],[47,18158,1166],{"className":18159},[508,581,545],[47,18161,616],{"className":18162},[615],[47,18164,18166],{"className":18165},[525],[47,18167,18169],{"className":18168,"style":17703},[529],[47,18170],{},[47,18172,13303],{"className":18173},[508],[47,18175,18177,18180],{"className":18176},[508],[47,18178,13164],{"className":18179},[508,581],[47,18181,18183],{"className":18182},[585],[47,18184,18186,18217],{"className":18185},[520,521],[47,18187,18189,18214],{"className":18188},[525],[47,18190,18192,18203],{"className":18191,"style":17670},[529],[47,18193,18194,18197],{"style":17673},[47,18195],{"className":18196,"style":929},[537],[47,18198,18200],{"className":18199},[542,543,544,545],[47,18201,724],{"className":18202},[508,581,545],[47,18204,18205,18208],{"style":16788},[47,18206],{"className":18207,"style":929},[537],[47,18209,18211],{"className":18210},[542,543,544,545],[47,18212,1166],{"className":18213},[508,581,545],[47,18215,616],{"className":18216},[615],[47,18218,18220],{"className":18219},[525],[47,18221,18223],{"className":18222,"style":17703},[529],[47,18224],{},[47,18226,1201],{"className":18227},[702],[47,18229,18230,18233],{"style":15717},[47,18231],{"className":18232,"style":15532},[537],[47,18234,18236,18239,18242,18245,18248,18310,18313,18377,18380,18484,18487,18490,18493,18522,18525,18528,18531,18536,18539,18579,18582,18633,18636,18687],{"className":18235},[508],[47,18237],{"className":18238},[508],[47,18240],{"className":18241,"style":889},[888],[47,18243,82],{"className":18244},[893],[47,18246],{"className":18247,"style":889},[888],[47,18249,18251,18254,18307],{"className":18250},[508],[47,18252],{"className":18253},[512,513],[47,18255,18257],{"className":18256},[467],[47,18258,18260,18299],{"className":18259},[520,521],[47,18261,18263,18296],{"className":18262},[525],[47,18264,18266,18277,18285],{"className":18265,"style":16641},[529],[47,18267,18268,18271],{"style":16181},[47,18269],{"className":18270,"style":538},[537],[47,18272,18274],{"className":18273},[508],[47,18275,7692],{"className":18276,"style":7767},[508,581],[47,18278,18279,18282],{"style":663},[47,18280],{"className":18281,"style":538},[537],[47,18283],{"className":18284,"style":671},[670],[47,18286,18287,18290],{"style":16247},[47,18288],{"className":18289,"style":538},[537],[47,18291,18293],{"className":18292},[508],[47,18294,472],{"className":18295},[508],[47,18297,616],{"className":18298},[615],[47,18300,18302],{"className":18301},[525],[47,18303,18305],{"className":18304,"style":16681},[529],[47,18306],{},[47,18308],{"className":18309},[702,513],[47,18311],{"className":18312,"style":1303},[888],[47,18314,18316],{"className":18315},[1338,14600],[47,18317,18319,18369],{"className":18318},[520,521],[47,18320,18322,18366],{"className":18321},[525],[47,18323,18325,18345,18355],{"className":18324,"style":16702},[529],[47,18326,18327,18330],{"style":16705},[47,18328],{"className":18329,"style":15794},[537],[47,18331,18333],{"className":18332},[542,543,544,545],[47,18334,18336,18339,18342],{"className":18335},[508,545],[47,18337,724],{"className":18338},[508,581,545],[47,18340,82],{"className":18341},[893,545],[47,18343,472],{"className":18344},[508,545],[47,18346,18347,18350],{"style":15803},[47,18348],{"className":18349,"style":15794},[537],[47,18351,18352],{},[47,18353,2213],{"className":18354},[1338,2297,15812],[47,18356,18357,18360],{"style":16736},[47,18358],{"className":18359,"style":15794},[537],[47,18361,18363],{"className":18362},[542,543,544,545],[47,18364,7692],{"className":18365,"style":7767},[508,581,545],[47,18367,616],{"className":18368},[615],[47,18370,18372],{"className":18371},[525],[47,18373,18375],{"className":18374,"style":16755},[529],[47,18376],{},[47,18378],{"className":18379,"style":1303},[888],[47,18381,18383],{"className":18382},[1338,14600],[47,18384,18386,18476],{"className":18385},[520,521],[47,18387,18389,18473],{"className":18388},[525],[47,18390,18392,18412,18422],{"className":18391,"style":17511},[529],[47,18393,18394,18397],{"style":16705},[47,18395],{"className":18396,"style":15794},[537],[47,18398,18400],{"className":18399},[542,543,544,545],[47,18401,18403,18406,18409],{"className":18402},[508,545],[47,18404,1166],{"className":18405},[508,581,545],[47,18407,82],{"className":18408},[893,545],[47,18410,472],{"className":18411},[508,545],[47,18413,18414,18417],{"style":15803},[47,18415],{"className":18416,"style":15794},[537],[47,18418,18419],{},[47,18420,2213],{"className":18421},[1338,2297,15812],[47,18423,18424,18427],{"style":17544},[47,18425],{"className":18426,"style":15794},[537],[47,18428,18430],{"className":18429},[542,543,544,545],[47,18431,18433],{"className":18432},[508,545],[47,18434,18436,18439],{"className":18435},[508,545],[47,18437,13604],{"className":18438,"style":3555},[508,581,545],[47,18440,18442],{"className":18441},[585],[47,18443,18445,18465],{"className":18444},[520,521],[47,18446,18448,18462],{"className":18447},[525],[47,18449,18451],{"className":18450,"style":4672},[529],[47,18452,18453,18456],{"style":17574},[47,18454],{"className":18455,"style":602},[537],[47,18457,18459],{"className":18458},[542,606,607,545],[47,18460,724],{"className":18461},[508,581,545],[47,18463,616],{"className":18464},[615],[47,18466,18468],{"className":18467},[525],[47,18469,18471],{"className":18470,"style":4694},[529],[47,18472],{},[47,18474,616],{"className":18475},[615],[47,18477,18479],{"className":18478},[525],[47,18480,18482],{"className":18481,"style":16755},[529],[47,18483],{},[47,18485],{"className":18486,"style":1303},[888],[47,18488,859],{"className":18489,"style":14471},[508,581],[47,18491,1187],{"className":18492},[512],[47,18494,18496,18499],{"className":18495},[508],[47,18497,13532],{"className":18498,"style":13548},[508,581],[47,18500,18502],{"className":18501},[585],[47,18503,18505],{"className":18504},[520],[47,18506,18508],{"className":18507},[525],[47,18509,18511],{"className":18510,"style":16785},[529],[47,18512,18513,18516],{"style":16788},[47,18514],{"className":18515,"style":929},[537],[47,18517,18519],{"className":18518},[542,543,544,545],[47,18520,724],{"className":18521},[508,581,545],[47,18523,1201],{"className":18524},[702],[47,18526,14742],{"className":18527},[508],[47,18529],{"className":18530,"style":1303},[888],[47,18532,17212,18534],{"className":18533},[1338],[47,18535,17216],{"style":17215},[47,18537],{"className":18538,"style":1303},[888],[47,18540,18542,18545],{"className":18541},[508],[47,18543,5974],{"className":18544,"style":3555},[508,581],[47,18546,18548],{"className":18547},[585],[47,18549,18551,18571],{"className":18550},[520,521],[47,18552,18554,18568],{"className":18553},[525],[47,18555,18557],{"className":18556,"style":1635},[529],[47,18558,18559,18562],{"style":7830},[47,18560],{"className":18561,"style":929},[537],[47,18563,18565],{"className":18564},[542,543,544,545],[47,18566,14413],{"className":18567,"style":3580},[508,581,545],[47,18569,616],{"className":18570},[615],[47,18572,18574],{"className":18573},[525],[47,18575,18577],{"className":18576,"style":1657},[529],[47,18578],{},[47,18580,1187],{"className":18581},[512],[47,18583,18585,18588],{"className":18584},[508],[47,18586,285],{"className":18587},[508,581],[47,18589,18591],{"className":18590},[585],[47,18592,18594,18625],{"className":18593},[520,521],[47,18595,18597,18622],{"className":18596},[525],[47,18598,18600,18611],{"className":18599,"style":17670},[529],[47,18601,18602,18605],{"style":17673},[47,18603],{"className":18604,"style":929},[537],[47,18606,18608],{"className":18607},[542,543,544,545],[47,18609,724],{"className":18610},[508,581,545],[47,18612,18613,18616],{"style":16788},[47,18614],{"className":18615,"style":929},[537],[47,18617,18619],{"className":18618},[542,543,544,545],[47,18620,1166],{"className":18621},[508,581,545],[47,18623,616],{"className":18624},[615],[47,18626,18628],{"className":18627},[525],[47,18629,18631],{"className":18630,"style":17703},[529],[47,18632],{},[47,18634,13303],{"className":18635},[508],[47,18637,18639,18642],{"className":18638},[508],[47,18640,13164],{"className":18641},[508,581],[47,18643,18645],{"className":18644},[585],[47,18646,18648,18679],{"className":18647},[520,521],[47,18649,18651,18676],{"className":18650},[525],[47,18652,18654,18665],{"className":18653,"style":17670},[529],[47,18655,18656,18659],{"style":17673},[47,18657],{"className":18658,"style":929},[537],[47,18660,18662],{"className":18661},[542,543,544,545],[47,18663,724],{"className":18664},[508,581,545],[47,18666,18667,18670],{"style":16788},[47,18668],{"className":18669,"style":929},[537],[47,18671,18673],{"className":18672},[542,543,544,545],[47,18674,1166],{"className":18675},[508,581,545],[47,18677,616],{"className":18678},[615],[47,18680,18682],{"className":18681},[525],[47,18683,18685],{"className":18684,"style":17703},[529],[47,18686],{},[47,18688,1201],{"className":18689},[702],[47,18691,616],{"className":18692},[615],[47,18694,18696],{"className":18695},[525],[47,18697,18699],{"className":18698,"style":15733},[529],[47,18700],{},[14,18702,18703],{},[47,18704,18706,18813],{"className":18705},[451],[47,18707,18709],{"className":18708},[455],[457,18710,18711],{"xmlns":459},[461,18712,18713,18810],{},[464,18714,18715,18717,18719,18721,18723,18725,18728,18734,18748,18766,18768,18770,18776,18778,18780,18782,18788,18790,18798,18800,18808],{},[480,18716,8186],{},[480,18718,1181],{},[480,18720,13164],{},[480,18722,13164],{},[850,18724,82],{},[850,18726,18727],{},"−",[467,18729,18730,18732],{},[470,18731,472],{},[480,18733,7692],{},[2209,18735,18736,18738,18746],{},[850,18737,2213],{},[464,18739,18740,18742,18744],{},[480,18741,724],{},[850,18743,82],{},[470,18745,472],{},[480,18747,7692],{},[2209,18749,18750,18752,18760],{},[850,18751,2213],{},[464,18753,18754,18756,18758],{},[480,18755,1166],{},[850,18757,82],{},[470,18759,472],{},[477,18761,18762,18764],{},[480,18763,13604],{},[480,18765,724],{},[480,18767,859],{},[850,18769,1187],{"stretchy":1186},[854,18771,18772,18774],{},[480,18773,13532],{},[480,18775,724],{},[850,18777,1201],{"stretchy":1186},[480,18779,15167],{},[850,18781,1210],{},[477,18783,18784,18786],{},[480,18785,5974],{},[480,18787,14413],{},[850,18789,1187],{"stretchy":1186},[2209,18791,18792,18794,18796],{},[480,18793,285],{},[480,18795,724],{},[480,18797,1166],{},[480,18799,13303],{"mathvariant":1206},[2209,18801,18802,18804,18806],{},[480,18803,13164],{},[480,18805,724],{},[480,18807,1166],{},[850,18809,1201],{"stretchy":1186},[487,18811,18812],{"encoding":489},"Loss = -\\frac{1}{N}\\sum\\limits_{n=1}^N \\sum\\limits_{t=1}^{T_n} R(\\tau^n) \\log P_\\theta(a_n^t|s_n^t)",[47,18814,18816,18838],{"className":18815,"ariaHidden":495},[494],[47,18817,18819,18822,18825,18829,18832,18835],{"className":18818},[499],[47,18820],{"className":18821,"style":14365},[503],[47,18823,8186],{"className":18824},[508,581],[47,18826,18828],{"className":18827},[508,581],"oss",[47,18830],{"className":18831,"style":889},[888],[47,18833,82],{"className":18834},[893],[47,18836],{"className":18837,"style":889},[888],[47,18839,18841,18845,18848,18918,18921,18989,18992,19098,19101,19104,19107,19136,19139,19142,19147,19150,19190,19193,19245,19248,19299],{"className":18840},[499],[47,18842],{"className":18843,"style":18844},[503],"height:2.5065em;vertical-align:-0.9671em;",[47,18846,18727],{"className":18847},[508],[47,18849,18851,18854,18915],{"className":18850},[508],[47,18852],{"className":18853},[512,513],[47,18855,18857],{"className":18856},[467],[47,18858,18860,18906],{"className":18859},[520,521],[47,18861,18863,18903],{"className":18862},[525],[47,18864,18866,18881,18889],{"className":18865,"style":530},[529],[47,18867,18869,18872],{"style":18868},"top:-2.655em;",[47,18870],{"className":18871,"style":538},[537],[47,18873,18875],{"className":18874},[542,543,544,545],[47,18876,18878],{"className":18877},[508,545],[47,18879,7692],{"className":18880,"style":7767},[508,581,545],[47,18882,18883,18886],{"style":663},[47,18884],{"className":18885,"style":538},[537],[47,18887],{"className":18888,"style":671},[670],[47,18890,18891,18894],{"style":674},[47,18892],{"className":18893,"style":538},[537],[47,18895,18897],{"className":18896},[542,543,544,545],[47,18898,18900],{"className":18899},[508,545],[47,18901,472],{"className":18902},[508,545],[47,18904,616],{"className":18905},[615],[47,18907,18909],{"className":18908},[525],[47,18910,18913],{"className":18911,"style":18912},[529],"height:0.345em;",[47,18914],{},[47,18916],{"className":18917},[702,513],[47,18919],{"className":18920,"style":1303},[888],[47,18922,18924],{"className":18923},[1338,14600],[47,18925,18927,18980],{"className":18926},[520,521],[47,18928,18930,18977],{"className":18929},[525],[47,18931,18934,18955,18965],{"className":18932,"style":18933},[529],"height:1.5283em;",[47,18935,18937,18940],{"style":18936},"top:-2.1329em;margin-left:0em;",[47,18938],{"className":18939,"style":538},[537],[47,18941,18943],{"className":18942},[542,543,544,545],[47,18944,18946,18949,18952],{"className":18945},[508,545],[47,18947,724],{"className":18948},[508,581,545],[47,18950,82],{"className":18951},[893,545],[47,18953,472],{"className":18954},[508,545],[47,18956,18957,18960],{"style":567},[47,18958],{"className":18959,"style":538},[537],[47,18961,18962],{},[47,18963,2213],{"className":18964},[1338,2297,2298],[47,18966,18968,18971],{"style":18967},"top:-3.95em;margin-left:0em;",[47,18969],{"className":18970,"style":538},[537],[47,18972,18974],{"className":18973},[542,543,544,545],[47,18975,7692],{"className":18976,"style":7767},[508,581,545],[47,18978,616],{"className":18979},[615],[47,18981,18983],{"className":18982},[525],[47,18984,18987],{"className":18985,"style":18986},[529],"height:0.9671em;",[47,18988],{},[47,18990],{"className":18991,"style":1303},[888],[47,18993,18995],{"className":18994},[1338,14600],[47,18996,18998,19090],{"className":18997},[520,521],[47,18999,19001,19087],{"className":19000},[525],[47,19002,19005,19025,19035],{"className":19003,"style":19004},[529],"height:1.5394em;",[47,19006,19007,19010],{"style":18936},[47,19008],{"className":19009,"style":538},[537],[47,19011,19013],{"className":19012},[542,543,544,545],[47,19014,19016,19019,19022],{"className":19015},[508,545],[47,19017,1166],{"className":19018},[508,581,545],[47,19020,82],{"className":19021},[893,545],[47,19023,472],{"className":19024},[508,545],[47,19026,19027,19030],{"style":567},[47,19028],{"className":19029,"style":538},[537],[47,19031,19032],{},[47,19033,2213],{"className":19034},[1338,2297,2298],[47,19036,19038,19041],{"style":19037},"top:-3.9611em;margin-left:0em;",[47,19039],{"className":19040,"style":538},[537],[47,19042,19044],{"className":19043},[542,543,544,545],[47,19045,19047],{"className":19046},[508,545],[47,19048,19050,19053],{"className":19049},[508,545],[47,19051,13604],{"className":19052,"style":3555},[508,581,545],[47,19054,19056],{"className":19055},[585],[47,19057,19059,19079],{"className":19058},[520,521],[47,19060,19062,19076],{"className":19061},[525],[47,19063,19065],{"className":19064,"style":4672},[529],[47,19066,19067,19070],{"style":17574},[47,19068],{"className":19069,"style":602},[537],[47,19071,19073],{"className":19072},[542,606,607,545],[47,19074,724],{"className":19075},[508,581,545],[47,19077,616],{"className":19078},[615],[47,19080,19082],{"className":19081},[525],[47,19083,19085],{"className":19084,"style":4694},[529],[47,19086],{},[47,19088,616],{"className":19089},[615],[47,19091,19093],{"className":19092},[525],[47,19094,19096],{"className":19095,"style":18986},[529],[47,19097],{},[47,19099],{"className":19100,"style":1303},[888],[47,19102,859],{"className":19103,"style":14471},[508,581],[47,19105,1187],{"className":19106},[512],[47,19108,19110,19113],{"className":19109},[508],[47,19111,13532],{"className":19112,"style":13548},[508,581],[47,19114,19116],{"className":19115},[585],[47,19117,19119],{"className":19118},[520],[47,19120,19122],{"className":19121},[525],[47,19123,19125],{"className":19124,"style":16984},[529],[47,19126,19127,19130],{"style":925},[47,19128],{"className":19129,"style":929},[537],[47,19131,19133],{"className":19132},[542,543,544,545],[47,19134,724],{"className":19135},[508,581,545],[47,19137,1201],{"className":19138},[702],[47,19140],{"className":19141,"style":1303},[888],[47,19143,17212,19145],{"className":19144},[1338],[47,19146,17216],{"style":17215},[47,19148],{"className":19149,"style":1303},[888],[47,19151,19153,19156],{"className":19152},[508],[47,19154,5974],{"className":19155,"style":3555},[508,581],[47,19157,19159],{"className":19158},[585],[47,19160,19162,19182],{"className":19161},[520,521],[47,19163,19165,19179],{"className":19164},[525],[47,19166,19168],{"className":19167,"style":1635},[529],[47,19169,19170,19173],{"style":7830},[47,19171],{"className":19172,"style":929},[537],[47,19174,19176],{"className":19175},[542,543,544,545],[47,19177,14413],{"className":19178,"style":3580},[508,581,545],[47,19180,616],{"className":19181},[615],[47,19183,19185],{"className":19184},[525],[47,19186,19188],{"className":19187,"style":1657},[529],[47,19189],{},[47,19191,1187],{"className":19192},[512],[47,19194,19196,19199],{"className":19195},[508],[47,19197,285],{"className":19198},[508,581],[47,19200,19202],{"className":19201},[585],[47,19203,19205,19237],{"className":19204},[520,521],[47,19206,19208,19234],{"className":19207},[525],[47,19209,19212,19223],{"className":19210,"style":19211},[529],"height:0.7936em;",[47,19213,19214,19217],{"style":17673},[47,19215],{"className":19216,"style":929},[537],[47,19218,19220],{"className":19219},[542,543,544,545],[47,19221,724],{"className":19222},[508,581,545],[47,19224,19225,19228],{"style":925},[47,19226],{"className":19227,"style":929},[537],[47,19229,19231],{"className":19230},[542,543,544,545],[47,19232,1166],{"className":19233},[508,581,545],[47,19235,616],{"className":19236},[615],[47,19238,19240],{"className":19239},[525],[47,19241,19243],{"className":19242,"style":17703},[529],[47,19244],{},[47,19246,13303],{"className":19247},[508],[47,19249,19251,19254],{"className":19250},[508],[47,19252,13164],{"className":19253},[508,581],[47,19255,19257],{"className":19256},[585],[47,19258,19260,19291],{"className":19259},[520,521],[47,19261,19263,19288],{"className":19262},[525],[47,19264,19266,19277],{"className":19265,"style":19211},[529],[47,19267,19268,19271],{"style":17673},[47,19269],{"className":19270,"style":929},[537],[47,19272,19274],{"className":19273},[542,543,544,545],[47,19275,724],{"className":19276},[508,581,545],[47,19278,19279,19282],{"style":925},[47,19280],{"className":19281,"style":929},[537],[47,19283,19285],{"className":19284},[542,543,544,545],[47,19286,1166],{"className":19287},[508,581,545],[47,19289,616],{"className":19290},[615],[47,19292,19294],{"className":19293},[525],[47,19295,19297],{"className":19296,"style":17703},[529],[47,19298],{},[47,19300,1201],{"className":19301},[702],[14,19303,19304],{},[47,19305,19307,19395],{"className":19306},[451],[47,19308,19310],{"className":19309},[455],[457,19311,19312],{"xmlns":459},[461,19313,19314,19392],{},[464,19315,19316,19318,19320,19326,19328,19330,19353,19370,19382,19384],{},[480,19317,859],{},[850,19319,1187],{"stretchy":1186},[854,19321,19322,19324],{},[480,19323,13532],{},[480,19325,724],{},[850,19327,1201],{"stretchy":1186},[850,19329,82],{},[2209,19331,19332,19334,19347],{},[850,19333,2213],{},[464,19335,19336,19338,19340],{},[480,19337,1166],{},[850,19339,82],{},[854,19341,19342,19344],{},[480,19343,1166],{},[850,19345,19346],{"mathvariant":1206,"lspace":14728,"rspace":14728},"′",[477,19348,19349,19351],{},[480,19350,13604],{},[480,19352,724],{},[854,19354,19355,19358],{},[480,19356,19357],{},"γ",[464,19359,19360,19362,19364],{},[480,19361,1166],{},[850,19363,18727],{},[854,19365,19366,19368],{},[480,19367,1166],{},[850,19369,19346],{"mathvariant":1206,"lspace":14728,"rspace":14728},[2209,19371,19372,19374,19380],{},[480,19373,8196],{},[854,19375,19376,19378],{},[480,19377,1166],{},[850,19379,19346],{"mathvariant":1206,"lspace":14728,"rspace":14728},[480,19381,724],{},[850,19383,82],{},[2209,19385,19386,19388,19390],{},[480,19387,859],{},[480,19389,1166],{},[480,19391,724],{},[487,19393,19394],{"encoding":489},"R(\\tau^n) = \\sum\\limits_{t=t'}^{T_n} \\gamma^{t-t'} r_{t'}^n = R_t^n",[47,19396,19398,19451,19762],{"className":19397,"ariaHidden":495},[494],[47,19399,19401,19404,19407,19410,19439,19442,19445,19448],{"className":19400},[499],[47,19402],{"className":19403,"style":1264},[503],[47,19405,859],{"className":19406,"style":14471},[508,581],[47,19408,1187],{"className":19409},[512],[47,19411,19413,19416],{"className":19412},[508],[47,19414,13532],{"className":19415,"style":13548},[508,581],[47,19417,19419],{"className":19418},[585],[47,19420,19422],{"className":19421},[520],[47,19423,19425],{"className":19424},[525],[47,19426,19428],{"className":19427,"style":16984},[529],[47,19429,19430,19433],{"style":925},[47,19431],{"className":19432,"style":929},[537],[47,19434,19436],{"className":19435},[542,543,544,545],[47,19437,724],{"className":19438},[508,581,545],[47,19440,1201],{"className":19441},[702],[47,19443],{"className":19444,"style":889},[888],[47,19446,82],{"className":19447},[893],[47,19449],{"className":19450,"style":889},[888],[47,19452,19454,19458,19595,19598,19668,19753,19756,19759],{"className":19453},[499],[47,19455],{"className":19456,"style":19457},[503],"height:2.5334em;vertical-align:-0.994em;",[47,19459,19461],{"className":19460},[1338,14600],[47,19462,19464,19586],{"className":19463},[520,521],[47,19465,19467,19583],{"className":19466},[525],[47,19468,19470,19522,19532],{"className":19469,"style":19004},[529],[47,19471,19473,19476],{"style":19472},"top:-2.106em;margin-left:0em;",[47,19474],{"className":19475,"style":538},[537],[47,19477,19479],{"className":19478},[542,543,544,545],[47,19480,19482,19485,19488],{"className":19481},[508,545],[47,19483,1166],{"className":19484},[508,581,545],[47,19486,82],{"className":19487},[893,545],[47,19489,19491,19494],{"className":19490},[508,545],[47,19492,1166],{"className":19493},[508,581,545],[47,19495,19497],{"className":19496},[585],[47,19498,19500],{"className":19499},[520],[47,19501,19503],{"className":19502},[525],[47,19504,19507],{"className":19505,"style":19506},[529],"height:0.6828em;",[47,19508,19510,19513],{"style":19509},"top:-2.786em;margin-right:0.0714em;",[47,19511],{"className":19512,"style":602},[537],[47,19514,19516],{"className":19515},[542,606,607,545],[47,19517,19519],{"className":19518},[508,545],[47,19520,19346],{"className":19521},[508,545],[47,19523,19524,19527],{"style":567},[47,19525],{"className":19526,"style":538},[537],[47,19528,19529],{},[47,19530,2213],{"className":19531},[1338,2297,2298],[47,19533,19534,19537],{"style":19037},[47,19535],{"className":19536,"style":538},[537],[47,19538,19540],{"className":19539},[542,543,544,545],[47,19541,19543],{"className":19542},[508,545],[47,19544,19546,19549],{"className":19545},[508,545],[47,19547,13604],{"className":19548,"style":3555},[508,581,545],[47,19550,19552],{"className":19551},[585],[47,19553,19555,19575],{"className":19554},[520,521],[47,19556,19558,19572],{"className":19557},[525],[47,19559,19561],{"className":19560,"style":4672},[529],[47,19562,19563,19566],{"style":17574},[47,19564],{"className":19565,"style":602},[537],[47,19567,19569],{"className":19568},[542,606,607,545],[47,19570,724],{"className":19571},[508,581,545],[47,19573,616],{"className":19574},[615],[47,19576,19578],{"className":19577},[525],[47,19579,19581],{"className":19580,"style":4694},[529],[47,19582],{},[47,19584,616],{"className":19585},[615],[47,19587,19589],{"className":19588},[525],[47,19590,19593],{"className":19591,"style":19592},[529],"height:0.994em;",[47,19594],{},[47,19596],{"className":19597,"style":1303},[888],[47,19599,19601,19605],{"className":19600},[508],[47,19602,19357],{"className":19603,"style":19604},[508,581],"margin-right:0.05556em;",[47,19606,19608],{"className":19607},[585],[47,19609,19611],{"className":19610},[520],[47,19612,19614],{"className":19613},[525],[47,19615,19618],{"className":19616,"style":19617},[529],"height:0.9425em;",[47,19619,19620,19623],{"style":925},[47,19621],{"className":19622,"style":929},[537],[47,19624,19626],{"className":19625},[542,543,544,545],[47,19627,19629,19632,19635],{"className":19628},[508,545],[47,19630,1166],{"className":19631},[508,581,545],[47,19633,18727],{"className":19634},[942,545],[47,19636,19638,19641],{"className":19637},[508,545],[47,19639,1166],{"className":19640},[508,581,545],[47,19642,19644],{"className":19643},[585],[47,19645,19647],{"className":19646},[520],[47,19648,19650],{"className":19649},[525],[47,19651,19654],{"className":19652,"style":19653},[529],"height:0.8278em;",[47,19655,19656,19659],{"style":1485},[47,19657],{"className":19658,"style":602},[537],[47,19660,19662],{"className":19661},[542,606,607,545],[47,19663,19665],{"className":19664},[508,545],[47,19666,19346],{"className":19667},[508,545],[47,19669,19671,19674],{"className":19670},[508],[47,19672,8196],{"className":19673,"style":3580},[508,581],[47,19675,19677],{"className":19676},[585],[47,19678,19680,19744],{"className":19679},[520,521],[47,19681,19683,19741],{"className":19682},[525],[47,19684,19686,19730],{"className":19685,"style":16984},[529],[47,19687,19689,19692],{"style":19688},"top:-2.425em;margin-left:-0.0278em;margin-right:0.05em;",[47,19690],{"className":19691,"style":929},[537],[47,19693,19695],{"className":19694},[542,543,544,545],[47,19696,19698],{"className":19697},[508,545],[47,19699,19701,19704],{"className":19700},[508,545],[47,19702,1166],{"className":19703},[508,581,545],[47,19705,19707],{"className":19706},[585],[47,19708,19710],{"className":19709},[520],[47,19711,19713],{"className":19712},[525],[47,19714,19716],{"className":19715,"style":19506},[529],[47,19717,19718,19721],{"style":19509},[47,19719],{"className":19720,"style":602},[537],[47,19722,19724],{"className":19723},[542,606,607,545],[47,19725,19727],{"className":19726},[508,545],[47,19728,19346],{"className":19729},[508,545],[47,19731,19732,19735],{"style":925},[47,19733],{"className":19734,"style":929},[537],[47,19736,19738],{"className":19737},[542,543,544,545],[47,19739,724],{"className":19740},[508,581,545],[47,19742,616],{"className":19743},[615],[47,19745,19747],{"className":19746},[525],[47,19748,19751],{"className":19749,"style":19750},[529],"height:0.275em;",[47,19752],{},[47,19754],{"className":19755,"style":889},[888],[47,19757,82],{"className":19758},[893],[47,19760],{"className":19761,"style":889},[888],[47,19763,19765,19769],{"className":19764},[499],[47,19766],{"className":19767,"style":19768},[503],"height:0.9303em;vertical-align:-0.247em;",[47,19770,19772,19775],{"className":19771},[508],[47,19773,859],{"className":19774,"style":14471},[508,581],[47,19776,19778],{"className":19777},[585],[47,19779,19781,19813],{"className":19780},[520,521],[47,19782,19784,19810],{"className":19783},[525],[47,19785,19787,19799],{"className":19786,"style":16984},[529],[47,19788,19790,19793],{"style":19789},"top:-2.453em;margin-left:-0.0077em;margin-right:0.05em;",[47,19791],{"className":19792,"style":929},[537],[47,19794,19796],{"className":19795},[542,543,544,545],[47,19797,1166],{"className":19798},[508,581,545],[47,19800,19801,19804],{"style":925},[47,19802],{"className":19803,"style":929},[537],[47,19805,19807],{"className":19806},[542,543,544,545],[47,19808,724],{"className":19809},[508,581,545],[47,19811,616],{"className":19812},[615],[47,19814,19816],{"className":19815},[525],[47,19817,19819],{"className":19818,"style":17703},[529],[47,19820],{},[10,19822,19824],{"id":19823},"actor-critic-演员-评论家算法","Actor-Critic 演员-评论家算法",[14,19826,19827,19828,19899],{},"为了衡量每个动作的相对好坏的情况，我们令奖励 ",[47,19829,19831,19849],{"className":19830},[451],[47,19832,19834],{"className":19833},[455],[457,19835,19836],{"xmlns":459},[461,19837,19838,19846],{},[464,19839,19840],{},[477,19841,19842,19844],{},[480,19843,859],{},[480,19845,1166],{},[487,19847,19848],{"encoding":489},"R_t",[47,19850,19852],{"className":19851,"ariaHidden":495},[494],[47,19853,19855,19858],{"className":19854},[499],[47,19856],{"className":19857,"style":7344},[503],[47,19859,19861,19864],{"className":19860},[508],[47,19862,859],{"className":19863,"style":14471},[508,581],[47,19865,19867],{"className":19866},[585],[47,19868,19870,19891],{"className":19869},[520,521],[47,19871,19873,19888],{"className":19872},[525],[47,19874,19876],{"className":19875,"style":13094},[529],[47,19877,19879,19882],{"style":19878},"top:-2.55em;margin-left:-0.0077em;margin-right:0.05em;",[47,19880],{"className":19881,"style":929},[537],[47,19883,19885],{"className":19884},[542,543,544,545],[47,19886,1166],{"className":19887},[508,581,545],[47,19889,616],{"className":19890},[615],[47,19892,19894],{"className":19893},[525],[47,19895,19897],{"className":19896,"style":1657},[529],[47,19898],{}," 减去一个基准值。",[14,19901,19902],{},[47,19903,19905,20018],{"className":19904},[451],[47,19906,19908],{"className":19907},[455],[457,19909,19910],{"xmlns":459},[461,19911,19912,20015],{},[464,19913,19914,19920,19934,19952,19954,19962,19964,19967,19969,19977,19979,19981,19983,19985,19987,19993,19995,20003,20005,20013],{},[467,19915,19916,19918],{},[470,19917,472],{},[480,19919,7692],{},[2209,19921,19922,19924,19932],{},[850,19923,2213],{},[464,19925,19926,19928,19930],{},[480,19927,724],{},[850,19929,82],{},[470,19931,472],{},[480,19933,7692],{},[2209,19935,19936,19938,19946],{},[850,19937,2213],{},[464,19939,19940,19942,19944],{},[480,19941,1166],{},[850,19943,82],{},[470,19945,472],{},[477,19947,19948,19950],{},[480,19949,13604],{},[480,19951,724],{},[850,19953,1187],{"stretchy":1186},[2209,19955,19956,19958,19960],{},[480,19957,859],{},[480,19959,1166],{},[480,19961,724],{},[850,19963,18727],{},[480,19965,19966],{},"B",[850,19968,1187],{"stretchy":1186},[2209,19970,19971,19973,19975],{},[480,19972,13164],{},[480,19974,1166],{},[480,19976,724],{},[850,19978,1201],{"stretchy":1186},[850,19980,1201],{"stretchy":1186},[480,19982,14742],{"mathvariant":1206},[480,19984,15167],{},[850,19986,1210],{},[477,19988,19989,19991],{},[480,19990,5974],{},[480,19992,14413],{},[850,19994,1187],{"stretchy":1186},[2209,19996,19997,19999,20001],{},[480,19998,285],{},[480,20000,1166],{},[480,20002,724],{},[480,20004,13303],{"mathvariant":1206},[2209,20006,20007,20009,20011],{},[480,20008,13164],{},[480,20010,1166],{},[480,20012,724],{},[850,20014,1201],{"stretchy":1186},[487,20016,20017],{"encoding":489}," \\frac{1}{N} \\sum\\limits_{n=1}^N \\sum\\limits_{t=1}^{T_n} (R_t^n - B(s_t^n))\\nabla \\log P_\\theta(a_t^n|s_t^n)",[47,20019,20021,20332],{"className":20020,"ariaHidden":495},[494],[47,20022,20024,20027,20095,20098,20162,20165,20269,20272,20323,20326,20329],{"className":20023},[499],[47,20025],{"className":20026,"style":18844},[503],[47,20028,20030,20033,20092],{"className":20029},[508],[47,20031],{"className":20032},[512,513],[47,20034,20036],{"className":20035},[467],[47,20037,20039,20084],{"className":20038},[520,521],[47,20040,20042,20081],{"className":20041},[525],[47,20043,20045,20059,20067],{"className":20044,"style":530},[529],[47,20046,20047,20050],{"style":18868},[47,20048],{"className":20049,"style":538},[537],[47,20051,20053],{"className":20052},[542,543,544,545],[47,20054,20056],{"className":20055},[508,545],[47,20057,7692],{"className":20058,"style":7767},[508,581,545],[47,20060,20061,20064],{"style":663},[47,20062],{"className":20063,"style":538},[537],[47,20065],{"className":20066,"style":671},[670],[47,20068,20069,20072],{"style":674},[47,20070],{"className":20071,"style":538},[537],[47,20073,20075],{"className":20074},[542,543,544,545],[47,20076,20078],{"className":20077},[508,545],[47,20079,472],{"className":20080},[508,545],[47,20082,616],{"className":20083},[615],[47,20085,20087],{"className":20086},[525],[47,20088,20090],{"className":20089,"style":18912},[529],[47,20091],{},[47,20093],{"className":20094},[702,513],[47,20096],{"className":20097,"style":1303},[888],[47,20099,20101],{"className":20100},[1338,14600],[47,20102,20104,20154],{"className":20103},[520,521],[47,20105,20107,20151],{"className":20106},[525],[47,20108,20110,20130,20140],{"className":20109,"style":18933},[529],[47,20111,20112,20115],{"style":18936},[47,20113],{"className":20114,"style":538},[537],[47,20116,20118],{"className":20117},[542,543,544,545],[47,20119,20121,20124,20127],{"className":20120},[508,545],[47,20122,724],{"className":20123},[508,581,545],[47,20125,82],{"className":20126},[893,545],[47,20128,472],{"className":20129},[508,545],[47,20131,20132,20135],{"style":567},[47,20133],{"className":20134,"style":538},[537],[47,20136,20137],{},[47,20138,2213],{"className":20139},[1338,2297,2298],[47,20141,20142,20145],{"style":18967},[47,20143],{"className":20144,"style":538},[537],[47,20146,20148],{"className":20147},[542,543,544,545],[47,20149,7692],{"className":20150,"style":7767},[508,581,545],[47,20152,616],{"className":20153},[615],[47,20155,20157],{"className":20156},[525],[47,20158,20160],{"className":20159,"style":18986},[529],[47,20161],{},[47,20163],{"className":20164,"style":1303},[888],[47,20166,20168],{"className":20167},[1338,14600],[47,20169,20171,20261],{"className":20170},[520,521],[47,20172,20174,20258],{"className":20173},[525],[47,20175,20177,20197,20207],{"className":20176,"style":19004},[529],[47,20178,20179,20182],{"style":18936},[47,20180],{"className":20181,"style":538},[537],[47,20183,20185],{"className":20184},[542,543,544,545],[47,20186,20188,20191,20194],{"className":20187},[508,545],[47,20189,1166],{"className":20190},[508,581,545],[47,20192,82],{"className":20193},[893,545],[47,20195,472],{"className":20196},[508,545],[47,20198,20199,20202],{"style":567},[47,20200],{"className":20201,"style":538},[537],[47,20203,20204],{},[47,20205,2213],{"className":20206},[1338,2297,2298],[47,20208,20209,20212],{"style":19037},[47,20210],{"className":20211,"style":538},[537],[47,20213,20215],{"className":20214},[542,543,544,545],[47,20216,20218],{"className":20217},[508,545],[47,20219,20221,20224],{"className":20220},[508,545],[47,20222,13604],{"className":20223,"style":3555},[508,581,545],[47,20225,20227],{"className":20226},[585],[47,20228,20230,20250],{"className":20229},[520,521],[47,20231,20233,20247],{"className":20232},[525],[47,20234,20236],{"className":20235,"style":4672},[529],[47,20237,20238,20241],{"style":17574},[47,20239],{"className":20240,"style":602},[537],[47,20242,20244],{"className":20243},[542,606,607,545],[47,20245,724],{"className":20246},[508,581,545],[47,20248,616],{"className":20249},[615],[47,20251,20253],{"className":20252},[525],[47,20254,20256],{"className":20255,"style":4694},[529],[47,20257],{},[47,20259,616],{"className":20260},[615],[47,20262,20264],{"className":20263},[525],[47,20265,20267],{"className":20266,"style":18986},[529],[47,20268],{},[47,20270,1187],{"className":20271},[512],[47,20273,20275,20278],{"className":20274},[508],[47,20276,859],{"className":20277,"style":14471},[508,581],[47,20279,20281],{"className":20280},[585],[47,20282,20284,20315],{"className":20283},[520,521],[47,20285,20287,20312],{"className":20286},[525],[47,20288,20290,20301],{"className":20289,"style":16984},[529],[47,20291,20292,20295],{"style":19789},[47,20293],{"className":20294,"style":929},[537],[47,20296,20298],{"className":20297},[542,543,544,545],[47,20299,1166],{"className":20300},[508,581,545],[47,20302,20303,20306],{"style":925},[47,20304],{"className":20305,"style":929},[537],[47,20307,20309],{"className":20308},[542,543,544,545],[47,20310,724],{"className":20311},[508,581,545],[47,20313,616],{"className":20314},[615],[47,20316,20318],{"className":20317},[525],[47,20319,20321],{"className":20320,"style":17703},[529],[47,20322],{},[47,20324],{"className":20325,"style":2259},[888],[47,20327,18727],{"className":20328},[942],[47,20330],{"className":20331,"style":2259},[888],[47,20333,20335,20338,20342,20345,20396,20399,20402,20405,20410,20413,20453,20456,20507,20510,20561],{"className":20334},[499],[47,20336],{"className":20337,"style":1264},[503],[47,20339,19966],{"className":20340,"style":20341},[508,581],"margin-right:0.05017em;",[47,20343,1187],{"className":20344},[512],[47,20346,20348,20351],{"className":20347},[508],[47,20349,13164],{"className":20350},[508,581],[47,20352,20354],{"className":20353},[585],[47,20355,20357,20388],{"className":20356},[520,521],[47,20358,20360,20385],{"className":20359},[525],[47,20361,20363,20374],{"className":20362,"style":16984},[529],[47,20364,20365,20368],{"style":17673},[47,20366],{"className":20367,"style":929},[537],[47,20369,20371],{"className":20370},[542,543,544,545],[47,20372,1166],{"className":20373},[508,581,545],[47,20375,20376,20379],{"style":925},[47,20377],{"className":20378,"style":929},[537],[47,20380,20382],{"className":20381},[542,543,544,545],[47,20383,724],{"className":20384},[508,581,545],[47,20386,616],{"className":20387},[615],[47,20389,20391],{"className":20390},[525],[47,20392,20394],{"className":20393,"style":17703},[529],[47,20395],{},[47,20397,8294],{"className":20398},[702],[47,20400,14742],{"className":20401},[508],[47,20403],{"className":20404,"style":1303},[888],[47,20406,17212,20408],{"className":20407},[1338],[47,20409,17216],{"style":17215},[47,20411],{"className":20412,"style":1303},[888],[47,20414,20416,20419],{"className":20415},[508],[47,20417,5974],{"className":20418,"style":3555},[508,581],[47,20420,20422],{"className":20421},[585],[47,20423,20425,20445],{"className":20424},[520,521],[47,20426,20428,20442],{"className":20427},[525],[47,20429,20431],{"className":20430,"style":1635},[529],[47,20432,20433,20436],{"style":7830},[47,20434],{"className":20435,"style":929},[537],[47,20437,20439],{"className":20438},[542,543,544,545],[47,20440,14413],{"className":20441,"style":3580},[508,581,545],[47,20443,616],{"className":20444},[615],[47,20446,20448],{"className":20447},[525],[47,20449,20451],{"className":20450,"style":1657},[529],[47,20452],{},[47,20454,1187],{"className":20455},[512],[47,20457,20459,20462],{"className":20458},[508],[47,20460,285],{"className":20461},[508,581],[47,20463,20465],{"className":20464},[585],[47,20466,20468,20499],{"className":20467},[520,521],[47,20469,20471,20496],{"className":20470},[525],[47,20472,20474,20485],{"className":20473,"style":16984},[529],[47,20475,20476,20479],{"style":17673},[47,20477],{"className":20478,"style":929},[537],[47,20480,20482],{"className":20481},[542,543,544,545],[47,20483,1166],{"className":20484},[508,581,545],[47,20486,20487,20490],{"style":925},[47,20488],{"className":20489,"style":929},[537],[47,20491,20493],{"className":20492},[542,543,544,545],[47,20494,724],{"className":20495},[508,581,545],[47,20497,616],{"className":20498},[615],[47,20500,20502],{"className":20501},[525],[47,20503,20505],{"className":20504,"style":17703},[529],[47,20506],{},[47,20508,13303],{"className":20509},[508],[47,20511,20513,20516],{"className":20512},[508],[47,20514,13164],{"className":20515},[508,581],[47,20517,20519],{"className":20518},[585],[47,20520,20522,20553],{"className":20521},[520,521],[47,20523,20525,20550],{"className":20524},[525],[47,20526,20528,20539],{"className":20527,"style":16984},[529],[47,20529,20530,20533],{"style":17673},[47,20531],{"className":20532,"style":929},[537],[47,20534,20536],{"className":20535},[542,543,544,545],[47,20537,1166],{"className":20538},[508,581,545],[47,20540,20541,20544],{"style":925},[47,20542],{"className":20543,"style":929},[537],[47,20545,20547],{"className":20546},[542,543,544,545],[47,20548,724],{"className":20549},[508,581,545],[47,20551,616],{"className":20552},[615],[47,20554,20556],{"className":20555},[525],[47,20557,20559],{"className":20558,"style":17703},[529],[47,20560],{},[47,20562,1201],{"className":20563},[702],[14,20565,20566,20567,20650,20651,20749,20750,20778,20779,20807],{},"但是，",[47,20568,20570,20590],{"className":20569},[451],[47,20571,20573],{"className":20572},[455],[457,20574,20575],{"xmlns":459},[461,20576,20577,20587],{},[464,20578,20579],{},[2209,20580,20581,20583,20585],{},[480,20582,859],{},[480,20584,1166],{},[480,20586,724],{},[487,20588,20589],{"encoding":489},"R_t^n",[47,20591,20593],{"className":20592,"ariaHidden":495},[494],[47,20594,20596,20599],{"className":20595},[499],[47,20597],{"className":20598,"style":19768},[503],[47,20600,20602,20605],{"className":20601},[508],[47,20603,859],{"className":20604,"style":14471},[508,581],[47,20606,20608],{"className":20607},[585],[47,20609,20611,20642],{"className":20610},[520,521],[47,20612,20614,20639],{"className":20613},[525],[47,20615,20617,20628],{"className":20616,"style":16984},[529],[47,20618,20619,20622],{"style":19789},[47,20620],{"className":20621,"style":929},[537],[47,20623,20625],{"className":20624},[542,543,544,545],[47,20626,1166],{"className":20627},[508,581,545],[47,20629,20630,20633],{"style":925},[47,20631],{"className":20632,"style":929},[537],[47,20634,20636],{"className":20635},[542,543,544,545],[47,20637,724],{"className":20638},[508,581,545],[47,20640,616],{"className":20641},[615],[47,20643,20645],{"className":20644},[525],[47,20646,20648],{"className":20647,"style":17703},[529],[47,20649],{}," 都是一次随机采样，方差很大，训练不稳定。我们引入 ",[47,20652,20654,20682],{"className":20653},[451],[47,20655,20657],{"className":20656},[455],[457,20658,20659],{"xmlns":459},[461,20660,20661,20679],{},[464,20662,20663,20669,20671,20673,20675,20677],{},[477,20664,20665,20667],{},[480,20666,848],{},[480,20668,14413],{},[850,20670,1187],{"stretchy":1186},[480,20672,13164],{},[850,20674,1192],{"separator":495},[480,20676,285],{},[850,20678,1201],{"stretchy":1186},[487,20680,20681],{"encoding":489},"Q_\\theta(s,a)",[47,20683,20685],{"className":20684,"ariaHidden":495},[494],[47,20686,20688,20691,20731,20734,20737,20740,20743,20746],{"className":20687},[499],[47,20689],{"className":20690,"style":1264},[503],[47,20692,20694,20697],{"className":20693},[508],[47,20695,848],{"className":20696},[508,581],[47,20698,20700],{"className":20699},[585],[47,20701,20703,20723],{"className":20702},[520,521],[47,20704,20706,20720],{"className":20705},[525],[47,20707,20709],{"className":20708,"style":1635},[529],[47,20710,20711,20714],{"style":1638},[47,20712],{"className":20713,"style":929},[537],[47,20715,20717],{"className":20716},[542,543,544,545],[47,20718,14413],{"className":20719,"style":3580},[508,581,545],[47,20721,616],{"className":20722},[615],[47,20724,20726],{"className":20725},[525],[47,20727,20729],{"className":20728,"style":1657},[529],[47,20730],{},[47,20732,1187],{"className":20733},[512],[47,20735,13164],{"className":20736},[508,581],[47,20738,1192],{"className":20739},[1299],[47,20741],{"className":20742,"style":1303},[888],[47,20744,285],{"className":20745},[508,581],[47,20747,1201],{"className":20748},[702],"，表示在状态 ",[47,20751,20753,20766],{"className":20752},[451],[47,20754,20756],{"className":20755},[455],[457,20757,20758],{"xmlns":459},[461,20759,20760,20764],{},[464,20761,20762],{},[480,20763,13164],{},[487,20765,13164],{"encoding":489},[47,20767,20769],{"className":20768,"ariaHidden":495},[494],[47,20770,20772,20775],{"className":20771},[499],[47,20773],{"className":20774,"style":736},[503],[47,20776,13164],{"className":20777},[508,581],"，下，做出动作 ",[47,20780,20782,20795],{"className":20781},[451],[47,20783,20785],{"className":20784},[455],[457,20786,20787],{"xmlns":459},[461,20788,20789,20793],{},[464,20790,20791],{},[480,20792,285],{},[487,20794,285],{"encoding":489},[47,20796,20798],{"className":20797,"ariaHidden":495},[494],[47,20799,20801,20804],{"className":20800},[499],[47,20802],{"className":20803,"style":736},[503],[47,20805,285],{"className":20806},[508,581]," 后期望的回报，即动作价值函数。",[14,20809,20810,20811,20897,20898,20926],{},"令 ",[47,20812,20814,20838],{"className":20813},[451],[47,20815,20817],{"className":20816},[455],[457,20818,20819],{"xmlns":459},[461,20820,20821,20835],{},[464,20822,20823,20829,20831,20833],{},[477,20824,20825,20827],{},[480,20826,1061],{},[480,20828,14413],{},[850,20830,1187],{"stretchy":1186},[480,20832,13164],{},[850,20834,1201],{"stretchy":1186},[487,20836,20837],{"encoding":489},"V_\\theta(s)",[47,20839,20841],{"className":20840,"ariaHidden":495},[494],[47,20842,20844,20847,20888,20891,20894],{"className":20843},[499],[47,20845],{"className":20846,"style":1264},[503],[47,20848,20850,20853],{"className":20849},[508],[47,20851,1061],{"className":20852,"style":1091},[508,581],[47,20854,20856],{"className":20855},[585],[47,20857,20859,20880],{"className":20858},[520,521],[47,20860,20862,20877],{"className":20861},[525],[47,20863,20865],{"className":20864,"style":1635},[529],[47,20866,20868,20871],{"style":20867},"top:-2.55em;margin-left:-0.2222em;margin-right:0.05em;",[47,20869],{"className":20870,"style":929},[537],[47,20872,20874],{"className":20873},[542,543,544,545],[47,20875,14413],{"className":20876,"style":3580},[508,581,545],[47,20878,616],{"className":20879},[615],[47,20881,20883],{"className":20882},[525],[47,20884,20886],{"className":20885,"style":1657},[529],[47,20887],{},[47,20889,1187],{"className":20890},[512],[47,20892,13164],{"className":20893},[508,581],[47,20895,1201],{"className":20896},[702]," 在状态 ",[47,20899,20901,20914],{"className":20900},[451],[47,20902,20904],{"className":20903},[455],[457,20905,20906],{"xmlns":459},[461,20907,20908,20912],{},[464,20909,20910],{},[480,20911,13164],{},[487,20913,13164],{"encoding":489},[47,20915,20917],{"className":20916,"ariaHidden":495},[494],[47,20918,20920,20923],{"className":20919},[499],[47,20921],{"className":20922,"style":736},[503],[47,20924,13164],{"className":20925},[508,581]," 下期望的回报，即状态价值函数",[14,20928,20929,21196,21197,21225,21226,21254],{},[47,20930,20932,20992],{"className":20931},[451],[47,20933,20935],{"className":20934},[455],[457,20936,20937],{"xmlns":459},[461,20938,20939,20989],{},[464,20940,20941,20947,20949,20951,20953,20955,20957,20959,20965,20967,20969,20971,20973,20975,20977,20983,20985,20987],{},[477,20942,20943,20945],{},[480,20944,1163],{},[480,20946,14413],{},[850,20948,1187],{"stretchy":1186},[480,20950,13164],{},[850,20952,1192],{"separator":495},[480,20954,285],{},[850,20956,1201],{"stretchy":1186},[850,20958,82],{},[477,20960,20961,20963],{},[480,20962,848],{},[480,20964,14413],{},[850,20966,1187],{"stretchy":1186},[480,20968,13164],{},[850,20970,1192],{"separator":495},[480,20972,285],{},[850,20974,1201],{"stretchy":1186},[850,20976,18727],{},[477,20978,20979,20981],{},[480,20980,1061],{},[480,20982,14413],{},[850,20984,1187],{"stretchy":1186},[480,20986,13164],{},[850,20988,1201],{"stretchy":1186},[487,20990,20991],{"encoding":489},"A_\\theta(s,a) = Q_\\theta(s,a) - V_\\theta(s)",[47,20993,20995,21068,21141],{"className":20994,"ariaHidden":495},[494],[47,20996,20998,21001,21041,21044,21047,21050,21053,21056,21059,21062,21065],{"className":20997},[499],[47,20999],{"className":21000,"style":1264},[503],[47,21002,21004,21007],{"className":21003},[508],[47,21005,1163],{"className":21006},[508,581],[47,21008,21010],{"className":21009},[585],[47,21011,21013,21033],{"className":21012},[520,521],[47,21014,21016,21030],{"className":21015},[525],[47,21017,21019],{"className":21018,"style":1635},[529],[47,21020,21021,21024],{"style":1638},[47,21022],{"className":21023,"style":929},[537],[47,21025,21027],{"className":21026},[542,543,544,545],[47,21028,14413],{"className":21029,"style":3580},[508,581,545],[47,21031,616],{"className":21032},[615],[47,21034,21036],{"className":21035},[525],[47,21037,21039],{"className":21038,"style":1657},[529],[47,21040],{},[47,21042,1187],{"className":21043},[512],[47,21045,13164],{"className":21046},[508,581],[47,21048,1192],{"className":21049},[1299],[47,21051],{"className":21052,"style":1303},[888],[47,21054,285],{"className":21055},[508,581],[47,21057,1201],{"className":21058},[702],[47,21060],{"className":21061,"style":889},[888],[47,21063,82],{"className":21064},[893],[47,21066],{"className":21067,"style":889},[888],[47,21069,21071,21074,21114,21117,21120,21123,21126,21129,21132,21135,21138],{"className":21070},[499],[47,21072],{"className":21073,"style":1264},[503],[47,21075,21077,21080],{"className":21076},[508],[47,21078,848],{"className":21079},[508,581],[47,21081,21083],{"className":21082},[585],[47,21084,21086,21106],{"className":21085},[520,521],[47,21087,21089,21103],{"className":21088},[525],[47,21090,21092],{"className":21091,"style":1635},[529],[47,21093,21094,21097],{"style":1638},[47,21095],{"className":21096,"style":929},[537],[47,21098,21100],{"className":21099},[542,543,544,545],[47,21101,14413],{"className":21102,"style":3580},[508,581,545],[47,21104,616],{"className":21105},[615],[47,21107,21109],{"className":21108},[525],[47,21110,21112],{"className":21111,"style":1657},[529],[47,21113],{},[47,21115,1187],{"className":21116},[512],[47,21118,13164],{"className":21119},[508,581],[47,21121,1192],{"className":21122},[1299],[47,21124],{"className":21125,"style":1303},[888],[47,21127,285],{"className":21128},[508,581],[47,21130,1201],{"className":21131},[702],[47,21133],{"className":21134,"style":2259},[888],[47,21136,18727],{"className":21137},[942],[47,21139],{"className":21140,"style":2259},[888],[47,21142,21144,21147,21187,21190,21193],{"className":21143},[499],[47,21145],{"className":21146,"style":1264},[503],[47,21148,21150,21153],{"className":21149},[508],[47,21151,1061],{"className":21152,"style":1091},[508,581],[47,21154,21156],{"className":21155},[585],[47,21157,21159,21179],{"className":21158},[520,521],[47,21160,21162,21176],{"className":21161},[525],[47,21163,21165],{"className":21164,"style":1635},[529],[47,21166,21167,21170],{"style":20867},[47,21168],{"className":21169,"style":929},[537],[47,21171,21173],{"className":21172},[542,543,544,545],[47,21174,14413],{"className":21175,"style":3580},[508,581,545],[47,21177,616],{"className":21178},[615],[47,21180,21182],{"className":21181},[525],[47,21183,21185],{"className":21184,"style":1657},[529],[47,21186],{},[47,21188,1187],{"className":21189},[512],[47,21191,13164],{"className":21192},[508,581],[47,21194,1201],{"className":21195},[702]," 表示优势函数（Advantage Function），衡量在状态 ",[47,21198,21200,21213],{"className":21199},[451],[47,21201,21203],{"className":21202},[455],[457,21204,21205],{"xmlns":459},[461,21206,21207,21211],{},[464,21208,21209],{},[480,21210,13164],{},[487,21212,13164],{"encoding":489},[47,21214,21216],{"className":21215,"ariaHidden":495},[494],[47,21217,21219,21222],{"className":21218},[499],[47,21220],{"className":21221,"style":736},[503],[47,21223,13164],{"className":21224},[508,581]," 下，动作 ",[47,21227,21229,21242],{"className":21228},[451],[47,21230,21232],{"className":21231},[455],[457,21233,21234],{"xmlns":459},[461,21235,21236,21240],{},[464,21237,21238],{},[480,21239,285],{},[487,21241,285],{"encoding":489},[47,21243,21245],{"className":21244,"ariaHidden":495},[494],[47,21246,21248,21251],{"className":21247},[499],[47,21249],{"className":21250,"style":736},[503],[47,21252,285],{"className":21253},[508,581]," 相对于平均水平的优势。",[14,21256,21257],{},"讲上述的公式改写为",[14,21259,21260],{},[47,21261,21263,21377],{"className":21262},[451],[47,21264,21266],{"className":21265},[455],[457,21267,21268],{"xmlns":459},[461,21269,21270,21374],{},[464,21271,21272,21280,21294,21312,21318,21320,21328,21330,21338,21340,21342,21344,21346,21352,21354,21362,21364,21372],{},[14736,21273,21274],{"scriptlevel":2141,"displaystyle":495},[467,21275,21276,21278],{},[470,21277,472],{},[480,21279,7692],{},[2209,21281,21282,21284,21292],{},[850,21283,2213],{},[464,21285,21286,21288,21290],{},[480,21287,724],{},[850,21289,82],{},[470,21291,472],{},[480,21293,7692],{},[2209,21295,21296,21298,21306],{},[850,21297,2213],{},[464,21299,21300,21302,21304],{},[480,21301,1166],{},[850,21303,82],{},[470,21305,472],{},[477,21307,21308,21310],{},[480,21309,13604],{},[480,21311,724],{},[477,21313,21314,21316],{},[480,21315,1163],{},[480,21317,14413],{},[850,21319,1187],{"stretchy":1186},[2209,21321,21322,21324,21326],{},[480,21323,13164],{},[480,21325,1166],{},[480,21327,724],{},[850,21329,1192],{"separator":495},[2209,21331,21332,21334,21336],{},[480,21333,285],{},[480,21335,1166],{},[480,21337,724],{},[850,21339,1201],{"stretchy":1186},[480,21341,14742],{"mathvariant":1206},[480,21343,15167],{},[850,21345,1210],{},[477,21347,21348,21350],{},[480,21349,5974],{},[480,21351,14413],{},[850,21353,1187],{"stretchy":1186},[2209,21355,21356,21358,21360],{},[480,21357,285],{},[480,21359,1166],{},[480,21361,724],{},[480,21363,13303],{"mathvariant":1206},[2209,21365,21366,21368,21370],{},[480,21367,13164],{},[480,21369,1166],{},[480,21371,724],{},[850,21373,1201],{"stretchy":1186},[487,21375,21376],{"encoding":489},"\\dfrac{1}{N}\\sum\\limits_{n=1}^N \\sum\\limits_{t=1}^{T_n} A_\\theta(s_t^n,a_t^n)\\nabla \\log P_\\theta(a_t^n|s_t^n)",[47,21378,21380],{"className":21379,"ariaHidden":495},[494],[47,21381,21383,21386,21448,21451,21515,21518,21622,21625,21665,21668,21719,21722,21725,21776,21779,21782,21785,21790,21793,21833,21836,21887,21890,21941],{"className":21382},[499],[47,21384],{"className":21385,"style":18844},[503],[47,21387,21389,21392,21445],{"className":21388},[508],[47,21390],{"className":21391},[512,513],[47,21393,21395],{"className":21394},[467],[47,21396,21398,21437],{"className":21397},[520,521],[47,21399,21401,21434],{"className":21400},[525],[47,21402,21404,21415,21423],{"className":21403,"style":16641},[529],[47,21405,21406,21409],{"style":16181},[47,21407],{"className":21408,"style":538},[537],[47,21410,21412],{"className":21411},[508],[47,21413,7692],{"className":21414,"style":7767},[508,581],[47,21416,21417,21420],{"style":663},[47,21418],{"className":21419,"style":538},[537],[47,21421],{"className":21422,"style":671},[670],[47,21424,21425,21428],{"style":16247},[47,21426],{"className":21427,"style":538},[537],[47,21429,21431],{"className":21430},[508],[47,21432,472],{"className":21433},[508],[47,21435,616],{"className":21436},[615],[47,21438,21440],{"className":21439},[525],[47,21441,21443],{"className":21442,"style":16681},[529],[47,21444],{},[47,21446],{"className":21447},[702,513],[47,21449],{"className":21450,"style":1303},[888],[47,21452,21454],{"className":21453},[1338,14600],[47,21455,21457,21507],{"className":21456},[520,521],[47,21458,21460,21504],{"className":21459},[525],[47,21461,21463,21483,21493],{"className":21462,"style":18933},[529],[47,21464,21465,21468],{"style":18936},[47,21466],{"className":21467,"style":538},[537],[47,21469,21471],{"className":21470},[542,543,544,545],[47,21472,21474,21477,21480],{"className":21473},[508,545],[47,21475,724],{"className":21476},[508,581,545],[47,21478,82],{"className":21479},[893,545],[47,21481,472],{"className":21482},[508,545],[47,21484,21485,21488],{"style":567},[47,21486],{"className":21487,"style":538},[537],[47,21489,21490],{},[47,21491,2213],{"className":21492},[1338,2297,2298],[47,21494,21495,21498],{"style":18967},[47,21496],{"className":21497,"style":538},[537],[47,21499,21501],{"className":21500},[542,543,544,545],[47,21502,7692],{"className":21503,"style":7767},[508,581,545],[47,21505,616],{"className":21506},[615],[47,21508,21510],{"className":21509},[525],[47,21511,21513],{"className":21512,"style":18986},[529],[47,21514],{},[47,21516],{"className":21517,"style":1303},[888],[47,21519,21521],{"className":21520},[1338,14600],[47,21522,21524,21614],{"className":21523},[520,521],[47,21525,21527,21611],{"className":21526},[525],[47,21528,21530,21550,21560],{"className":21529,"style":19004},[529],[47,21531,21532,21535],{"style":18936},[47,21533],{"className":21534,"style":538},[537],[47,21536,21538],{"className":21537},[542,543,544,545],[47,21539,21541,21544,21547],{"className":21540},[508,545],[47,21542,1166],{"className":21543},[508,581,545],[47,21545,82],{"className":21546},[893,545],[47,21548,472],{"className":21549},[508,545],[47,21551,21552,21555],{"style":567},[47,21553],{"className":21554,"style":538},[537],[47,21556,21557],{},[47,21558,2213],{"className":21559},[1338,2297,2298],[47,21561,21562,21565],{"style":19037},[47,21563],{"className":21564,"style":538},[537],[47,21566,21568],{"className":21567},[542,543,544,545],[47,21569,21571],{"className":21570},[508,545],[47,21572,21574,21577],{"className":21573},[508,545],[47,21575,13604],{"className":21576,"style":3555},[508,581,545],[47,21578,21580],{"className":21579},[585],[47,21581,21583,21603],{"className":21582},[520,521],[47,21584,21586,21600],{"className":21585},[525],[47,21587,21589],{"className":21588,"style":4672},[529],[47,21590,21591,21594],{"style":17574},[47,21592],{"className":21593,"style":602},[537],[47,21595,21597],{"className":21596},[542,606,607,545],[47,21598,724],{"className":21599},[508,581,545],[47,21601,616],{"className":21602},[615],[47,21604,21606],{"className":21605},[525],[47,21607,21609],{"className":21608,"style":4694},[529],[47,21610],{},[47,21612,616],{"className":21613},[615],[47,21615,21617],{"className":21616},[525],[47,21618,21620],{"className":21619,"style":18986},[529],[47,21621],{},[47,21623],{"className":21624,"style":1303},[888],[47,21626,21628,21631],{"className":21627},[508],[47,21629,1163],{"className":21630},[508,581],[47,21632,21634],{"className":21633},[585],[47,21635,21637,21657],{"className":21636},[520,521],[47,21638,21640,21654],{"className":21639},[525],[47,21641,21643],{"className":21642,"style":1635},[529],[47,21644,21645,21648],{"style":1638},[47,21646],{"className":21647,"style":929},[537],[47,21649,21651],{"className":21650},[542,543,544,545],[47,21652,14413],{"className":21653,"style":3580},[508,581,545],[47,21655,616],{"className":21656},[615],[47,21658,21660],{"className":21659},[525],[47,21661,21663],{"className":21662,"style":1657},[529],[47,21664],{},[47,21666,1187],{"className":21667},[512],[47,21669,21671,21674],{"className":21670},[508],[47,21672,13164],{"className":21673},[508,581],[47,21675,21677],{"className":21676},[585],[47,21678,21680,21711],{"className":21679},[520,521],[47,21681,21683,21708],{"className":21682},[525],[47,21684,21686,21697],{"className":21685,"style":16984},[529],[47,21687,21688,21691],{"style":17673},[47,21689],{"className":21690,"style":929},[537],[47,21692,21694],{"className":21693},[542,543,544,545],[47,21695,1166],{"className":21696},[508,581,545],[47,21698,21699,21702],{"style":925},[47,21700],{"className":21701,"style":929},[537],[47,21703,21705],{"className":21704},[542,543,544,545],[47,21706,724],{"className":21707},[508,581,545],[47,21709,616],{"className":21710},[615],[47,21712,21714],{"className":21713},[525],[47,21715,21717],{"className":21716,"style":17703},[529],[47,21718],{},[47,21720,1192],{"className":21721},[1299],[47,21723],{"className":21724,"style":1303},[888],[47,21726,21728,21731],{"className":21727},[508],[47,21729,285],{"className":21730},[508,581],[47,21732,21734],{"className":21733},[585],[47,21735,21737,21768],{"className":21736},[520,521],[47,21738,21740,21765],{"className":21739},[525],[47,21741,21743,21754],{"className":21742,"style":16984},[529],[47,21744,21745,21748],{"style":17673},[47,21746],{"className":21747,"style":929},[537],[47,21749,21751],{"className":21750},[542,543,544,545],[47,21752,1166],{"className":21753},[508,581,545],[47,21755,21756,21759],{"style":925},[47,21757],{"className":21758,"style":929},[537],[47,21760,21762],{"className":21761},[542,543,544,545],[47,21763,724],{"className":21764},[508,581,545],[47,21766,616],{"className":21767},[615],[47,21769,21771],{"className":21770},[525],[47,21772,21774],{"className":21773,"style":17703},[529],[47,21775],{},[47,21777,1201],{"className":21778},[702],[47,21780,14742],{"className":21781},[508],[47,21783],{"className":21784,"style":1303},[888],[47,21786,17212,21788],{"className":21787},[1338],[47,21789,17216],{"style":17215},[47,21791],{"className":21792,"style":1303},[888],[47,21794,21796,21799],{"className":21795},[508],[47,21797,5974],{"className":21798,"style":3555},[508,581],[47,21800,21802],{"className":21801},[585],[47,21803,21805,21825],{"className":21804},[520,521],[47,21806,21808,21822],{"className":21807},[525],[47,21809,21811],{"className":21810,"style":1635},[529],[47,21812,21813,21816],{"style":7830},[47,21814],{"className":21815,"style":929},[537],[47,21817,21819],{"className":21818},[542,543,544,545],[47,21820,14413],{"className":21821,"style":3580},[508,581,545],[47,21823,616],{"className":21824},[615],[47,21826,21828],{"className":21827},[525],[47,21829,21831],{"className":21830,"style":1657},[529],[47,21832],{},[47,21834,1187],{"className":21835},[512],[47,21837,21839,21842],{"className":21838},[508],[47,21840,285],{"className":21841},[508,581],[47,21843,21845],{"className":21844},[585],[47,21846,21848,21879],{"className":21847},[520,521],[47,21849,21851,21876],{"className":21850},[525],[47,21852,21854,21865],{"className":21853,"style":16984},[529],[47,21855,21856,21859],{"style":17673},[47,21857],{"className":21858,"style":929},[537],[47,21860,21862],{"className":21861},[542,543,544,545],[47,21863,1166],{"className":21864},[508,581,545],[47,21866,21867,21870],{"style":925},[47,21868],{"className":21869,"style":929},[537],[47,21871,21873],{"className":21872},[542,543,544,545],[47,21874,724],{"className":21875},[508,581,545],[47,21877,616],{"className":21878},[615],[47,21880,21882],{"className":21881},[525],[47,21883,21885],{"className":21884,"style":17703},[529],[47,21886],{},[47,21888,13303],{"className":21889},[508],[47,21891,21893,21896],{"className":21892},[508],[47,21894,13164],{"className":21895},[508,581],[47,21897,21899],{"className":21898},[585],[47,21900,21902,21933],{"className":21901},[520,521],[47,21903,21905,21930],{"className":21904},[525],[47,21906,21908,21919],{"className":21907,"style":16984},[529],[47,21909,21910,21913],{"style":17673},[47,21911],{"className":21912,"style":929},[537],[47,21914,21916],{"className":21915},[542,543,544,545],[47,21917,1166],{"className":21918},[508,581,545],[47,21920,21921,21924],{"style":925},[47,21922],{"className":21923,"style":929},[537],[47,21925,21927],{"className":21926},[542,543,544,545],[47,21928,724],{"className":21929},[508,581,545],[47,21931,616],{"className":21932},[615],[47,21934,21936],{"className":21935},[525],[47,21937,21939],{"className":21938,"style":17703},[529],[47,21940],{},[47,21942,1201],{"className":21943},[702],[47,21945,21947],{"className":21946},[14712],[47,21948,21950,22155],{"className":21949},[451],[47,21951,21953],{"className":21952},[455],[457,21954,21955],{"xmlns":459,"display":9848},[461,21956,21957,22152],{},[464,21958,21959,21965,21967,21973,21975,21977,21979,21981,21987,21989,21991,21997,21999,22011,22013,22016,22022,22024,22030,22032,22034,22036,22038,22044,22046,22048,22054,22056,22068,22070,22072,22078,22080,22086,22088,22090,22096,22098,22110,22112,22114,22126,22128,22130,22136,22138,22150],{},[477,21960,21961,21963],{},[480,21962,848],{},[480,21964,14413],{},[850,21966,1187],{"stretchy":1186},[477,21968,21969,21971],{},[480,21970,13164],{},[480,21972,1166],{},[850,21974,1192],{"separator":495},[480,21976,285],{},[850,21978,1201],{"stretchy":1186},[850,21980,82],{},[477,21982,21983,21985],{},[480,21984,8196],{},[480,21986,1166],{},[850,21988,6147],{},[480,21990,19357],{},[477,21992,21993,21995],{},[480,21994,1061],{},[480,21996,14413],{},[850,21998,1187],{"stretchy":1186},[477,22000,22001,22003],{},[480,22002,13164],{},[464,22004,22005,22007,22009],{},[480,22006,1166],{},[850,22008,6147],{},[470,22010,472],{},[850,22012,1201],{"stretchy":1186},[888,22014],{"linebreak":22015},"newline",[477,22017,22018,22020],{},[480,22019,1163],{},[480,22021,14413],{},[850,22023,1187],{"stretchy":1186},[477,22025,22026,22028],{},[480,22027,13164],{},[480,22029,1166],{},[850,22031,1192],{"separator":495},[480,22033,285],{},[850,22035,1201],{"stretchy":1186},[850,22037,82],{},[477,22039,22040,22042],{},[480,22041,8196],{},[480,22043,1166],{},[850,22045,6147],{},[480,22047,19357],{},[477,22049,22050,22052],{},[480,22051,1061],{},[480,22053,14413],{},[850,22055,1187],{"stretchy":1186},[477,22057,22058,22060],{},[480,22059,13164],{},[464,22061,22062,22064,22066],{},[480,22063,1166],{},[850,22065,6147],{},[470,22067,472],{},[850,22069,1201],{"stretchy":1186},[850,22071,18727],{},[477,22073,22074,22076],{},[480,22075,1061],{},[480,22077,14413],{},[850,22079,1187],{"stretchy":1186},[477,22081,22082,22084],{},[480,22083,13164],{},[480,22085,1166],{},[850,22087,1201],{"stretchy":1186},[888,22089],{"linebreak":22015},[477,22091,22092,22094],{},[480,22093,1061],{},[480,22095,14413],{},[850,22097,1187],{"stretchy":1186},[477,22099,22100,22102],{},[480,22101,13164],{},[464,22103,22104,22106,22108],{},[480,22105,1166],{},[850,22107,6147],{},[470,22109,472],{},[850,22111,1201],{"stretchy":1186},[850,22113,15037],{},[477,22115,22116,22118],{},[480,22117,8196],{},[464,22119,22120,22122,22124],{},[480,22121,1166],{},[850,22123,6147],{},[470,22125,472],{},[850,22127,6147],{},[480,22129,19357],{},[477,22131,22132,22134],{},[480,22133,1061],{},[480,22135,14413],{},[850,22137,1187],{"stretchy":1186},[477,22139,22140,22142],{},[480,22141,13164],{},[464,22143,22144,22146,22148],{},[480,22145,1166],{},[850,22147,6147],{},[470,22149,3192],{},[850,22151,1201],{"stretchy":1186},[487,22153,22154],{"encoding":489}," Q_\\theta(s_t, a) = r_t + \\gamma V_\\theta(s_{t+1}) \\\\\n\n A_\\theta(s_t, a) = r_t + \\gamma V_\\theta(s_{t+1}) - V_\\theta(s_t)\\\\\n V_\\theta(s_{t+1}) \\approx r_{t+1} + \\gamma V_\\theta(s_{t+2})",[47,22156,22158,22268,22324,22428,22431,22541,22596,22709,22801,22804,22914,22979],{"className":22157,"ariaHidden":495},[494],[47,22159,22161,22164,22204,22207,22247,22250,22253,22256,22259,22262,22265],{"className":22160},[499],[47,22162],{"className":22163,"style":1264},[503],[47,22165,22167,22170],{"className":22166},[508],[47,22168,848],{"className":22169},[508,581],[47,22171,22173],{"className":22172},[585],[47,22174,22176,22196],{"className":22175},[520,521],[47,22177,22179,22193],{"className":22178},[525],[47,22180,22182],{"className":22181,"style":1635},[529],[47,22183,22184,22187],{"style":1638},[47,22185],{"className":22186,"style":929},[537],[47,22188,22190],{"className":22189},[542,543,544,545],[47,22191,14413],{"className":22192,"style":3580},[508,581,545],[47,22194,616],{"className":22195},[615],[47,22197,22199],{"className":22198},[525],[47,22200,22202],{"className":22201,"style":1657},[529],[47,22203],{},[47,22205,1187],{"className":22206},[512],[47,22208,22210,22213],{"className":22209},[508],[47,22211,13164],{"className":22212},[508,581],[47,22214,22216],{"className":22215},[585],[47,22217,22219,22239],{"className":22218},[520,521],[47,22220,22222,22236],{"className":22221},[525],[47,22223,22225],{"className":22224,"style":13094},[529],[47,22226,22227,22230],{"style":1638},[47,22228],{"className":22229,"style":929},[537],[47,22231,22233],{"className":22232},[542,543,544,545],[47,22234,1166],{"className":22235},[508,581,545],[47,22237,616],{"className":22238},[615],[47,22240,22242],{"className":22241},[525],[47,22243,22245],{"className":22244,"style":1657},[529],[47,22246],{},[47,22248,1192],{"className":22249},[1299],[47,22251],{"className":22252,"style":1303},[888],[47,22254,285],{"className":22255},[508,581],[47,22257,1201],{"className":22258},[702],[47,22260],{"className":22261,"style":889},[888],[47,22263,82],{"className":22264},[893],[47,22266],{"className":22267,"style":889},[888],[47,22269,22271,22275,22315,22318,22321],{"className":22270},[499],[47,22272],{"className":22273,"style":22274},[503],"height:0.7333em;vertical-align:-0.15em;",[47,22276,22278,22281],{"className":22277},[508],[47,22279,8196],{"className":22280,"style":3580},[508,581],[47,22282,22284],{"className":22283},[585],[47,22285,22287,22307],{"className":22286},[520,521],[47,22288,22290,22304],{"className":22289},[525],[47,22291,22293],{"className":22292,"style":13094},[529],[47,22294,22295,22298],{"style":13467},[47,22296],{"className":22297,"style":929},[537],[47,22299,22301],{"className":22300},[542,543,544,545],[47,22302,1166],{"className":22303},[508,581,545],[47,22305,616],{"className":22306},[615],[47,22308,22310],{"className":22309},[525],[47,22311,22313],{"className":22312,"style":1657},[529],[47,22314],{},[47,22316],{"className":22317,"style":2259},[888],[47,22319,6147],{"className":22320},[942],[47,22322],{"className":22323,"style":2259},[888],[47,22325,22327,22330,22333,22373,22376,22425],{"className":22326},[499],[47,22328],{"className":22329,"style":1264},[503],[47,22331,19357],{"className":22332,"style":19604},[508,581],[47,22334,22336,22339],{"className":22335},[508],[47,22337,1061],{"className":22338,"style":1091},[508,581],[47,22340,22342],{"className":22341},[585],[47,22343,22345,22365],{"className":22344},[520,521],[47,22346,22348,22362],{"className":22347},[525],[47,22349,22351],{"className":22350,"style":1635},[529],[47,22352,22353,22356],{"style":20867},[47,22354],{"className":22355,"style":929},[537],[47,22357,22359],{"className":22358},[542,543,544,545],[47,22360,14413],{"className":22361,"style":3580},[508,581,545],[47,22363,616],{"className":22364},[615],[47,22366,22368],{"className":22367},[525],[47,22369,22371],{"className":22370,"style":1657},[529],[47,22372],{},[47,22374,1187],{"className":22375},[512],[47,22377,22379,22382],{"className":22378},[508],[47,22380,13164],{"className":22381},[508,581],[47,22383,22385],{"className":22384},[585],[47,22386,22388,22417],{"className":22387},[520,521],[47,22389,22391,22414],{"className":22390},[525],[47,22392,22394],{"className":22393,"style":3461},[529],[47,22395,22396,22399],{"style":1638},[47,22397],{"className":22398,"style":929},[537],[47,22400,22402],{"className":22401},[542,543,544,545],[47,22403,22405,22408,22411],{"className":22404},[508,545],[47,22406,1166],{"className":22407},[508,581,545],[47,22409,6147],{"className":22410},[942,545],[47,22412,472],{"className":22413},[508,545],[47,22415,616],{"className":22416},[615],[47,22418,22420],{"className":22419},[525],[47,22421,22423],{"className":22422,"style":7308},[529],[47,22424],{},[47,22426,1201],{"className":22427},[702],[47,22429],{"className":22430},[888,22015],[47,22432,22434,22437,22477,22480,22520,22523,22526,22529,22532,22535,22538],{"className":22433},[499],[47,22435],{"className":22436,"style":1264},[503],[47,22438,22440,22443],{"className":22439},[508],[47,22441,1163],{"className":22442},[508,581],[47,22444,22446],{"className":22445},[585],[47,22447,22449,22469],{"className":22448},[520,521],[47,22450,22452,22466],{"className":22451},[525],[47,22453,22455],{"className":22454,"style":1635},[529],[47,22456,22457,22460],{"style":1638},[47,22458],{"className":22459,"style":929},[537],[47,22461,22463],{"className":22462},[542,543,544,545],[47,22464,14413],{"className":22465,"style":3580},[508,581,545],[47,22467,616],{"className":22468},[615],[47,22470,22472],{"className":22471},[525],[47,22473,22475],{"className":22474,"style":1657},[529],[47,22476],{},[47,22478,1187],{"className":22479},[512],[47,22481,22483,22486],{"className":22482},[508],[47,22484,13164],{"className":22485},[508,581],[47,22487,22489],{"className":22488},[585],[47,22490,22492,22512],{"className":22491},[520,521],[47,22493,22495,22509],{"className":22494},[525],[47,22496,22498],{"className":22497,"style":13094},[529],[47,22499,22500,22503],{"style":1638},[47,22501],{"className":22502,"style":929},[537],[47,22504,22506],{"className":22505},[542,543,544,545],[47,22507,1166],{"className":22508},[508,581,545],[47,22510,616],{"className":22511},[615],[47,22513,22515],{"className":22514},[525],[47,22516,22518],{"className":22517,"style":1657},[529],[47,22519],{},[47,22521,1192],{"className":22522},[1299],[47,22524],{"className":22525,"style":1303},[888],[47,22527,285],{"className":22528},[508,581],[47,22530,1201],{"className":22531},[702],[47,22533],{"className":22534,"style":889},[888],[47,22536,82],{"className":22537},[893],[47,22539],{"className":22540,"style":889},[888],[47,22542,22544,22547,22587,22590,22593],{"className":22543},[499],[47,22545],{"className":22546,"style":22274},[503],[47,22548,22550,22553],{"className":22549},[508],[47,22551,8196],{"className":22552,"style":3580},[508,581],[47,22554,22556],{"className":22555},[585],[47,22557,22559,22579],{"className":22558},[520,521],[47,22560,22562,22576],{"className":22561},[525],[47,22563,22565],{"className":22564,"style":13094},[529],[47,22566,22567,22570],{"style":13467},[47,22568],{"className":22569,"style":929},[537],[47,22571,22573],{"className":22572},[542,543,544,545],[47,22574,1166],{"className":22575},[508,581,545],[47,22577,616],{"className":22578},[615],[47,22580,22582],{"className":22581},[525],[47,22583,22585],{"className":22584,"style":1657},[529],[47,22586],{},[47,22588],{"className":22589,"style":2259},[888],[47,22591,6147],{"className":22592},[942],[47,22594],{"className":22595,"style":2259},[888],[47,22597,22599,22602,22605,22645,22648,22697,22700,22703,22706],{"className":22598},[499],[47,22600],{"className":22601,"style":1264},[503],[47,22603,19357],{"className":22604,"style":19604},[508,581],[47,22606,22608,22611],{"className":22607},[508],[47,22609,1061],{"className":22610,"style":1091},[508,581],[47,22612,22614],{"className":22613},[585],[47,22615,22617,22637],{"className":22616},[520,521],[47,22618,22620,22634],{"className":22619},[525],[47,22621,22623],{"className":22622,"style":1635},[529],[47,22624,22625,22628],{"style":20867},[47,22626],{"className":22627,"style":929},[537],[47,22629,22631],{"className":22630},[542,543,544,545],[47,22632,14413],{"className":22633,"style":3580},[508,581,545],[47,22635,616],{"className":22636},[615],[47,22638,22640],{"className":22639},[525],[47,22641,22643],{"className":22642,"style":1657},[529],[47,22644],{},[47,22646,1187],{"className":22647},[512],[47,22649,22651,22654],{"className":22650},[508],[47,22652,13164],{"className":22653},[508,581],[47,22655,22657],{"className":22656},[585],[47,22658,22660,22689],{"className":22659},[520,521],[47,22661,22663,22686],{"className":22662},[525],[47,22664,22666],{"className":22665,"style":3461},[529],[47,22667,22668,22671],{"style":1638},[47,22669],{"className":22670,"style":929},[537],[47,22672,22674],{"className":22673},[542,543,544,545],[47,22675,22677,22680,22683],{"className":22676},[508,545],[47,22678,1166],{"className":22679},[508,581,545],[47,22681,6147],{"className":22682},[942,545],[47,22684,472],{"className":22685},[508,545],[47,22687,616],{"className":22688},[615],[47,22690,22692],{"className":22691},[525],[47,22693,22695],{"className":22694,"style":7308},[529],[47,22696],{},[47,22698,1201],{"className":22699},[702],[47,22701],{"className":22702,"style":2259},[888],[47,22704,18727],{"className":22705},[942],[47,22707],{"className":22708,"style":2259},[888],[47,22710,22712,22715,22755,22758,22798],{"className":22711},[499],[47,22713],{"className":22714,"style":1264},[503],[47,22716,22718,22721],{"className":22717},[508],[47,22719,1061],{"className":22720,"style":1091},[508,581],[47,22722,22724],{"className":22723},[585],[47,22725,22727,22747],{"className":22726},[520,521],[47,22728,22730,22744],{"className":22729},[525],[47,22731,22733],{"className":22732,"style":1635},[529],[47,22734,22735,22738],{"style":20867},[47,22736],{"className":22737,"style":929},[537],[47,22739,22741],{"className":22740},[542,543,544,545],[47,22742,14413],{"className":22743,"style":3580},[508,581,545],[47,22745,616],{"className":22746},[615],[47,22748,22750],{"className":22749},[525],[47,22751,22753],{"className":22752,"style":1657},[529],[47,22754],{},[47,22756,1187],{"className":22757},[512],[47,22759,22761,22764],{"className":22760},[508],[47,22762,13164],{"className":22763},[508,581],[47,22765,22767],{"className":22766},[585],[47,22768,22770,22790],{"className":22769},[520,521],[47,22771,22773,22787],{"className":22772},[525],[47,22774,22776],{"className":22775,"style":13094},[529],[47,22777,22778,22781],{"style":1638},[47,22779],{"className":22780,"style":929},[537],[47,22782,22784],{"className":22783},[542,543,544,545],[47,22785,1166],{"className":22786},[508,581,545],[47,22788,616],{"className":22789},[615],[47,22791,22793],{"className":22792},[525],[47,22794,22796],{"className":22795,"style":1657},[529],[47,22797],{},[47,22799,1201],{"className":22800},[702],[47,22802],{"className":22803},[888,22015],[47,22805,22807,22810,22850,22853,22902,22905,22908,22911],{"className":22806},[499],[47,22808],{"className":22809,"style":1264},[503],[47,22811,22813,22816],{"className":22812},[508],[47,22814,1061],{"className":22815,"style":1091},[508,581],[47,22817,22819],{"className":22818},[585],[47,22820,22822,22842],{"className":22821},[520,521],[47,22823,22825,22839],{"className":22824},[525],[47,22826,22828],{"className":22827,"style":1635},[529],[47,22829,22830,22833],{"style":20867},[47,22831],{"className":22832,"style":929},[537],[47,22834,22836],{"className":22835},[542,543,544,545],[47,22837,14413],{"className":22838,"style":3580},[508,581,545],[47,22840,616],{"className":22841},[615],[47,22843,22845],{"className":22844},[525],[47,22846,22848],{"className":22847,"style":1657},[529],[47,22849],{},[47,22851,1187],{"className":22852},[512],[47,22854,22856,22859],{"className":22855},[508],[47,22857,13164],{"className":22858},[508,581],[47,22860,22862],{"className":22861},[585],[47,22863,22865,22894],{"className":22864},[520,521],[47,22866,22868,22891],{"className":22867},[525],[47,22869,22871],{"className":22870,"style":3461},[529],[47,22872,22873,22876],{"style":1638},[47,22874],{"className":22875,"style":929},[537],[47,22877,22879],{"className":22878},[542,543,544,545],[47,22880,22882,22885,22888],{"className":22881},[508,545],[47,22883,1166],{"className":22884},[508,581,545],[47,22886,6147],{"className":22887},[942,545],[47,22889,472],{"className":22890},[508,545],[47,22892,616],{"className":22893},[615],[47,22895,22897],{"className":22896},[525],[47,22898,22900],{"className":22899,"style":7308},[529],[47,22901],{},[47,22903,1201],{"className":22904},[702],[47,22906],{"className":22907,"style":889},[888],[47,22909,15037],{"className":22910},[893],[47,22912],{"className":22913,"style":889},[888],[47,22915,22917,22921,22970,22973,22976],{"className":22916},[499],[47,22918],{"className":22919,"style":22920},[503],"height:0.7917em;vertical-align:-0.2083em;",[47,22922,22924,22927],{"className":22923},[508],[47,22925,8196],{"className":22926,"style":3580},[508,581],[47,22928,22930],{"className":22929},[585],[47,22931,22933,22962],{"className":22932},[520,521],[47,22934,22936,22959],{"className":22935},[525],[47,22937,22939],{"className":22938,"style":3461},[529],[47,22940,22941,22944],{"style":13467},[47,22942],{"className":22943,"style":929},[537],[47,22945,22947],{"className":22946},[542,543,544,545],[47,22948,22950,22953,22956],{"className":22949},[508,545],[47,22951,1166],{"className":22952},[508,581,545],[47,22954,6147],{"className":22955},[942,545],[47,22957,472],{"className":22958},[508,545],[47,22960,616],{"className":22961},[615],[47,22963,22965],{"className":22964},[525],[47,22966,22968],{"className":22967,"style":7308},[529],[47,22969],{},[47,22971],{"className":22972,"style":2259},[888],[47,22974,6147],{"className":22975},[942],[47,22977],{"className":22978,"style":2259},[888],[47,22980,22982,22985,22988,23028,23031,23080],{"className":22981},[499],[47,22983],{"className":22984,"style":1264},[503],[47,22986,19357],{"className":22987,"style":19604},[508,581],[47,22989,22991,22994],{"className":22990},[508],[47,22992,1061],{"className":22993,"style":1091},[508,581],[47,22995,22997],{"className":22996},[585],[47,22998,23000,23020],{"className":22999},[520,521],[47,23001,23003,23017],{"className":23002},[525],[47,23004,23006],{"className":23005,"style":1635},[529],[47,23007,23008,23011],{"style":20867},[47,23009],{"className":23010,"style":929},[537],[47,23012,23014],{"className":23013},[542,543,544,545],[47,23015,14413],{"className":23016,"style":3580},[508,581,545],[47,23018,616],{"className":23019},[615],[47,23021,23023],{"className":23022},[525],[47,23024,23026],{"className":23025,"style":1657},[529],[47,23027],{},[47,23029,1187],{"className":23030},[512],[47,23032,23034,23037],{"className":23033},[508],[47,23035,13164],{"className":23036},[508,581],[47,23038,23040],{"className":23039},[585],[47,23041,23043,23072],{"className":23042},[520,521],[47,23044,23046,23069],{"className":23045},[525],[47,23047,23049],{"className":23048,"style":3461},[529],[47,23050,23051,23054],{"style":1638},[47,23052],{"className":23053,"style":929},[537],[47,23055,23057],{"className":23056},[542,543,544,545],[47,23058,23060,23063,23066],{"className":23059},[508,545],[47,23061,1166],{"className":23062},[508,581,545],[47,23064,6147],{"className":23065},[942,545],[47,23067,3192],{"className":23068},[508,545],[47,23070,616],{"className":23071},[615],[47,23073,23075],{"className":23074},[525],[47,23076,23078],{"className":23077,"style":7308},[529],[47,23079],{},[47,23081,1201],{"className":23082},[702],[14,23084,23085,23086,23184],{},"通过上述公式变化，关于 ",[47,23087,23089,23117],{"className":23088},[451],[47,23090,23092],{"className":23091},[455],[457,23093,23094],{"xmlns":459},[461,23095,23096,23114],{},[464,23097,23098,23104,23106,23108,23110,23112],{},[477,23099,23100,23102],{},[480,23101,1163],{},[480,23103,14413],{},[850,23105,1187],{"stretchy":1186},[480,23107,13164],{},[850,23109,1192],{"separator":495},[480,23111,285],{},[850,23113,1201],{"stretchy":1186},[487,23115,23116],{"encoding":489},"A_\\theta(s,a)",[47,23118,23120],{"className":23119,"ariaHidden":495},[494],[47,23121,23123,23126,23166,23169,23172,23175,23178,23181],{"className":23122},[499],[47,23124],{"className":23125,"style":1264},[503],[47,23127,23129,23132],{"className":23128},[508],[47,23130,1163],{"className":23131},[508,581],[47,23133,23135],{"className":23134},[585],[47,23136,23138,23158],{"className":23137},[520,521],[47,23139,23141,23155],{"className":23140},[525],[47,23142,23144],{"className":23143,"style":1635},[529],[47,23145,23146,23149],{"style":1638},[47,23147],{"className":23148,"style":929},[537],[47,23150,23152],{"className":23151},[542,543,544,545],[47,23153,14413],{"className":23154,"style":3580},[508,581,545],[47,23156,616],{"className":23157},[615],[47,23159,23161],{"className":23160},[525],[47,23162,23164],{"className":23163,"style":1657},[529],[47,23165],{},[47,23167,1187],{"className":23168},[512],[47,23170,13164],{"className":23171},[508,581],[47,23173,1192],{"className":23174},[1299],[47,23176],{"className":23177,"style":1303},[888],[47,23179,285],{"className":23180},[508,581],[47,23182,1201],{"className":23183},[702]," 的推到公式仅剩一个相关变量，可以降低公式的复杂性。",[14,23186,23187],{},"之后再根据采样步数进行估算。采样越多，情况越多，整体估计的偏差就会偏小，同样的，碰到极值的概率越大，方差就越大。",[47,23189,23191],{"className":23190},[14712],[47,23192,23194,23644],{"className":23193},[451],[47,23195,23197],{"className":23196},[455],[457,23198,23199],{"xmlns":459,"display":9848},[461,23200,23201,23641],{},[464,23202,23203,23211,23213,23219,23221,23223,23225,23227,23233,23235,23237,23243,23245,23257,23259,23261,23267,23269,23275,23277,23279,23287,23289,23295,23297,23299,23301,23303,23309,23311,23313,23325,23327,23333,23339,23341,23353,23355,23357,23363,23365,23371,23373,23375,23383,23385,23391,23393,23395,23397,23399,23405,23407,23409,23412,23424,23426,23432,23434,23446,23448,23454,23460,23462,23474,23476,23478,23484,23486,23492,23494,23496,23508,23510,23518,23520,23526,23528,23530,23532,23534,23540,23542,23544,23546,23558,23560,23566,23568,23580,23582,23588,23590,23602,23604,23607,23609,23615,23617,23623,23625,23631,23633,23639],{},[2209,23204,23205,23207,23209],{},[480,23206,1163],{},[480,23208,14413],{},[470,23210,472],{},[850,23212,1187],{"stretchy":1186},[477,23214,23215,23217],{},[480,23216,13164],{},[480,23218,1166],{},[850,23220,1192],{"separator":495},[480,23222,285],{},[850,23224,1201],{"stretchy":1186},[850,23226,82],{},[477,23228,23229,23231],{},[480,23230,8196],{},[480,23232,1166],{},[850,23234,6147],{},[480,23236,19357],{},[477,23238,23239,23241],{},[480,23240,1061],{},[480,23242,14413],{},[850,23244,1187],{"stretchy":1186},[477,23246,23247,23249],{},[480,23248,13164],{},[464,23250,23251,23253,23255],{},[480,23252,1166],{},[850,23254,6147],{},[470,23256,472],{},[850,23258,1201],{"stretchy":1186},[850,23260,18727],{},[477,23262,23263,23265],{},[480,23264,1061],{},[480,23266,14413],{},[850,23268,1187],{"stretchy":1186},[477,23270,23271,23273],{},[480,23272,13164],{},[480,23274,1166],{},[850,23276,1201],{"stretchy":1186},[888,23278],{"linebreak":22015},[2209,23280,23281,23283,23285],{},[480,23282,1163],{},[480,23284,14413],{},[470,23286,3192],{},[850,23288,1187],{"stretchy":1186},[477,23290,23291,23293],{},[480,23292,13164],{},[480,23294,1166],{},[850,23296,1192],{"separator":495},[480,23298,285],{},[850,23300,1201],{"stretchy":1186},[850,23302,82],{},[477,23304,23305,23307],{},[480,23306,8196],{},[480,23308,1166],{},[850,23310,6147],{},[480,23312,19357],{},[477,23314,23315,23317],{},[480,23316,8196],{},[464,23318,23319,23321,23323],{},[480,23320,1166],{},[850,23322,6147],{},[470,23324,472],{},[850,23326,6147],{},[854,23328,23329,23331],{},[480,23330,19357],{},[470,23332,3192],{},[477,23334,23335,23337],{},[480,23336,1061],{},[480,23338,14413],{},[850,23340,1187],{"stretchy":1186},[477,23342,23343,23345],{},[480,23344,13164],{},[464,23346,23347,23349,23351],{},[480,23348,1166],{},[850,23350,6147],{},[470,23352,3192],{},[850,23354,1201],{"stretchy":1186},[850,23356,18727],{},[477,23358,23359,23361],{},[480,23360,1061],{},[480,23362,14413],{},[850,23364,1187],{"stretchy":1186},[477,23366,23367,23369],{},[480,23368,13164],{},[480,23370,1166],{},[850,23372,1201],{"stretchy":1186},[888,23374],{"linebreak":22015},[2209,23376,23377,23379,23381],{},[480,23378,1163],{},[480,23380,14413],{},[470,23382,5041],{},[850,23384,1187],{"stretchy":1186},[477,23386,23387,23389],{},[480,23388,13164],{},[480,23390,1166],{},[850,23392,1192],{"separator":495},[480,23394,285],{},[850,23396,1201],{"stretchy":1186},[850,23398,82],{},[477,23400,23401,23403],{},[480,23402,8196],{},[480,23404,1166],{},[850,23406,6147],{},[480,23408,19357],{},[850,23410,23411],{},"∗",[477,23413,23414,23416],{},[480,23415,8196],{},[464,23417,23418,23420,23422],{},[480,23419,1166],{},[850,23421,6147],{},[470,23423,472],{},[850,23425,6147],{},[854,23427,23428,23430],{},[480,23429,19357],{},[470,23431,3192],{},[850,23433,23411],{},[477,23435,23436,23438],{},[480,23437,8196],{},[464,23439,23440,23442,23444],{},[480,23441,1166],{},[850,23443,6147],{},[470,23445,3192],{},[850,23447,6147],{},[854,23449,23450,23452],{},[480,23451,19357],{},[470,23453,5041],{},[477,23455,23456,23458],{},[480,23457,1061],{},[480,23459,14413],{},[850,23461,1187],{"stretchy":1186},[477,23463,23464,23466],{},[480,23465,13164],{},[464,23467,23468,23470,23472],{},[480,23469,1166],{},[850,23471,6147],{},[470,23473,5041],{},[850,23475,1201],{"stretchy":1186},[850,23477,18727],{},[477,23479,23480,23482],{},[480,23481,1061],{},[480,23483,14413],{},[850,23485,1187],{"stretchy":1186},[477,23487,23488,23490],{},[480,23489,13164],{},[480,23491,1166],{},[850,23493,1201],{"stretchy":1186},[888,23495],{"linebreak":22015},[464,23497,23498,23501],{},[480,23499,23500],{"mathvariant":1206},"⋮",[23502,23503,23504],"mpadded",{"height":14728,"voffset":14728},[888,23505],{"mathbackground":23506,"width":14728,"height":23507},"black","1.5em",[888,23509],{"linebreak":22015},[2209,23511,23512,23514,23516],{},[480,23513,1163],{},[480,23515,14413],{},[480,23517,13604],{},[850,23519,1187],{"stretchy":1186},[477,23521,23522,23524],{},[480,23523,13164],{},[480,23525,1166],{},[850,23527,1192],{"separator":495},[480,23529,285],{},[850,23531,1201],{"stretchy":1186},[850,23533,82],{},[477,23535,23536,23538],{},[480,23537,8196],{},[480,23539,1166],{},[850,23541,6147],{},[480,23543,19357],{},[850,23545,23411],{},[477,23547,23548,23550],{},[480,23549,8196],{},[464,23551,23552,23554,23556],{},[480,23553,1166],{},[850,23555,6147],{},[470,23557,472],{},[850,23559,6147],{},[854,23561,23562,23564],{},[480,23563,19357],{},[470,23565,3192],{},[850,23567,23411],{},[477,23569,23570,23572],{},[480,23571,8196],{},[464,23573,23574,23576,23578],{},[480,23575,1166],{},[850,23577,6147],{},[470,23579,3192],{},[850,23581,6147],{},[854,23583,23584,23586],{},[480,23585,19357],{},[470,23587,5041],{},[850,23589,23411],{},[477,23591,23592,23594],{},[480,23593,8196],{},[464,23595,23596,23598,23600],{},[480,23597,1166],{},[850,23599,6147],{},[470,23601,5041],{},[850,23603,6147],{},[850,23605,23606],{},"⋯",[850,23608,6147],{},[854,23610,23611,23613],{},[480,23612,19357],{},[480,23614,13604],{},[850,23616,23411],{},[477,23618,23619,23621],{},[480,23620,8196],{},[480,23622,13604],{},[850,23624,18727],{},[477,23626,23627,23629],{},[480,23628,1061],{},[480,23630,14413],{},[850,23632,1187],{"stretchy":1186},[477,23634,23635,23637],{},[480,23636,13164],{},[480,23638,1166],{},[850,23640,1201],{"stretchy":1186},[487,23642,23643],{"encoding":489}," A_\\theta^1(s_t, a) = r_t + \\gamma V_\\theta(s_{t+1}) - V_\\theta(s_t)\\\\\n A_\\theta^2(s_t, a) = r_t + \\gamma r_{t+1} + \\gamma^2 V_\\theta(s_{t+2}) - V_\\theta(s_t)\\\\\n A_\\theta^3(s_t, a) = r_t + \\gamma * r_{t+1} + \\gamma^2 * r_{t+2} + \\gamma^3 V_\\theta(s_{t+3}) - V_\\theta(s_t)\\\\\n \\vdots \\\\\n A_\\theta^T(s_t, a) = r_t + \\gamma * r_{t+1} + \\gamma^2 * r_{t+2} + \\gamma^3 * r_{t+3} + \\cdots + \\gamma^T * r_T - V_\\theta(s_t)",[47,23645,23647,23770,23825,23938,24030,24033,24154,24209,24276,24415,24507,24510,24631,24686,24705,24769,24814,24878,25017,25109,25112,25130,25133,25256,25311,25329,25393,25437,25501,25545,25609,25628,25673,25728],{"className":23646,"ariaHidden":495},[494],[47,23648,23650,23654,23706,23709,23749,23752,23755,23758,23761,23764,23767],{"className":23649},[499],[47,23651],{"className":23652,"style":23653},[503],"height:1.1141em;vertical-align:-0.25em;",[47,23655,23657,23660],{"className":23656},[508],[47,23658,1163],{"className":23659},[508,581],[47,23661,23663],{"className":23662},[585],[47,23664,23666,23698],{"className":23665},[520,521],[47,23667,23669,23695],{"className":23668},[525],[47,23670,23673,23684],{"className":23671,"style":23672},[529],"height:0.8641em;",[47,23674,23675,23678],{"style":17673},[47,23676],{"className":23677,"style":929},[537],[47,23679,23681],{"className":23680},[542,543,544,545],[47,23682,14413],{"className":23683,"style":3580},[508,581,545],[47,23685,23686,23689],{"style":16788},[47,23687],{"className":23688,"style":929},[537],[47,23690,23692],{"className":23691},[542,543,544,545],[47,23693,472],{"className":23694},[508,545],[47,23696,616],{"className":23697},[615],[47,23699,23701],{"className":23700},[525],[47,23702,23704],{"className":23703,"style":17703},[529],[47,23705],{},[47,23707,1187],{"className":23708},[512],[47,23710,23712,23715],{"className":23711},[508],[47,23713,13164],{"className":23714},[508,581],[47,23716,23718],{"className":23717},[585],[47,23719,23721,23741],{"className":23720},[520,521],[47,23722,23724,23738],{"className":23723},[525],[47,23725,23727],{"className":23726,"style":13094},[529],[47,23728,23729,23732],{"style":1638},[47,23730],{"className":23731,"style":929},[537],[47,23733,23735],{"className":23734},[542,543,544,545],[47,23736,1166],{"className":23737},[508,581,545],[47,23739,616],{"className":23740},[615],[47,23742,23744],{"className":23743},[525],[47,23745,23747],{"className":23746,"style":1657},[529],[47,23748],{},[47,23750,1192],{"className":23751},[1299],[47,23753],{"className":23754,"style":1303},[888],[47,23756,285],{"className":23757},[508,581],[47,23759,1201],{"className":23760},[702],[47,23762],{"className":23763,"style":889},[888],[47,23765,82],{"className":23766},[893],[47,23768],{"className":23769,"style":889},[888],[47,23771,23773,23776,23816,23819,23822],{"className":23772},[499],[47,23774],{"className":23775,"style":22274},[503],[47,23777,23779,23782],{"className":23778},[508],[47,23780,8196],{"className":23781,"style":3580},[508,581],[47,23783,23785],{"className":23784},[585],[47,23786,23788,23808],{"className":23787},[520,521],[47,23789,23791,23805],{"className":23790},[525],[47,23792,23794],{"className":23793,"style":13094},[529],[47,23795,23796,23799],{"style":13467},[47,23797],{"className":23798,"style":929},[537],[47,23800,23802],{"className":23801},[542,543,544,545],[47,23803,1166],{"className":23804},[508,581,545],[47,23806,616],{"className":23807},[615],[47,23809,23811],{"className":23810},[525],[47,23812,23814],{"className":23813,"style":1657},[529],[47,23815],{},[47,23817],{"className":23818,"style":2259},[888],[47,23820,6147],{"className":23821},[942],[47,23823],{"className":23824,"style":2259},[888],[47,23826,23828,23831,23834,23874,23877,23926,23929,23932,23935],{"className":23827},[499],[47,23829],{"className":23830,"style":1264},[503],[47,23832,19357],{"className":23833,"style":19604},[508,581],[47,23835,23837,23840],{"className":23836},[508],[47,23838,1061],{"className":23839,"style":1091},[508,581],[47,23841,23843],{"className":23842},[585],[47,23844,23846,23866],{"className":23845},[520,521],[47,23847,23849,23863],{"className":23848},[525],[47,23850,23852],{"className":23851,"style":1635},[529],[47,23853,23854,23857],{"style":20867},[47,23855],{"className":23856,"style":929},[537],[47,23858,23860],{"className":23859},[542,543,544,545],[47,23861,14413],{"className":23862,"style":3580},[508,581,545],[47,23864,616],{"className":23865},[615],[47,23867,23869],{"className":23868},[525],[47,23870,23872],{"className":23871,"style":1657},[529],[47,23873],{},[47,23875,1187],{"className":23876},[512],[47,23878,23880,23883],{"className":23879},[508],[47,23881,13164],{"className":23882},[508,581],[47,23884,23886],{"className":23885},[585],[47,23887,23889,23918],{"className":23888},[520,521],[47,23890,23892,23915],{"className":23891},[525],[47,23893,23895],{"className":23894,"style":3461},[529],[47,23896,23897,23900],{"style":1638},[47,23898],{"className":23899,"style":929},[537],[47,23901,23903],{"className":23902},[542,543,544,545],[47,23904,23906,23909,23912],{"className":23905},[508,545],[47,23907,1166],{"className":23908},[508,581,545],[47,23910,6147],{"className":23911},[942,545],[47,23913,472],{"className":23914},[508,545],[47,23916,616],{"className":23917},[615],[47,23919,23921],{"className":23920},[525],[47,23922,23924],{"className":23923,"style":7308},[529],[47,23925],{},[47,23927,1201],{"className":23928},[702],[47,23930],{"className":23931,"style":2259},[888],[47,23933,18727],{"className":23934},[942],[47,23936],{"className":23937,"style":2259},[888],[47,23939,23941,23944,23984,23987,24027],{"className":23940},[499],[47,23942],{"className":23943,"style":1264},[503],[47,23945,23947,23950],{"className":23946},[508],[47,23948,1061],{"className":23949,"style":1091},[508,581],[47,23951,23953],{"className":23952},[585],[47,23954,23956,23976],{"className":23955},[520,521],[47,23957,23959,23973],{"className":23958},[525],[47,23960,23962],{"className":23961,"style":1635},[529],[47,23963,23964,23967],{"style":20867},[47,23965],{"className":23966,"style":929},[537],[47,23968,23970],{"className":23969},[542,543,544,545],[47,23971,14413],{"className":23972,"style":3580},[508,581,545],[47,23974,616],{"className":23975},[615],[47,23977,23979],{"className":23978},[525],[47,23980,23982],{"className":23981,"style":1657},[529],[47,23983],{},[47,23985,1187],{"className":23986},[512],[47,23988,23990,23993],{"className":23989},[508],[47,23991,13164],{"className":23992},[508,581],[47,23994,23996],{"className":23995},[585],[47,23997,23999,24019],{"className":23998},[520,521],[47,24000,24002,24016],{"className":24001},[525],[47,24003,24005],{"className":24004,"style":13094},[529],[47,24006,24007,24010],{"style":1638},[47,24008],{"className":24009,"style":929},[537],[47,24011,24013],{"className":24012},[542,543,544,545],[47,24014,1166],{"className":24015},[508,581,545],[47,24017,616],{"className":24018},[615],[47,24020,24022],{"className":24021},[525],[47,24023,24025],{"className":24024,"style":1657},[529],[47,24026],{},[47,24028,1201],{"className":24029},[702],[47,24031],{"className":24032},[888,22015],[47,24034,24036,24039,24090,24093,24133,24136,24139,24142,24145,24148,24151],{"className":24035},[499],[47,24037],{"className":24038,"style":23653},[503],[47,24040,24042,24045],{"className":24041},[508],[47,24043,1163],{"className":24044},[508,581],[47,24046,24048],{"className":24047},[585],[47,24049,24051,24082],{"className":24050},[520,521],[47,24052,24054,24079],{"className":24053},[525],[47,24055,24057,24068],{"className":24056,"style":23672},[529],[47,24058,24059,24062],{"style":17673},[47,24060],{"className":24061,"style":929},[537],[47,24063,24065],{"className":24064},[542,543,544,545],[47,24066,14413],{"className":24067,"style":3580},[508,581,545],[47,24069,24070,24073],{"style":16788},[47,24071],{"className":24072,"style":929},[537],[47,24074,24076],{"className":24075},[542,543,544,545],[47,24077,3192],{"className":24078},[508,545],[47,24080,616],{"className":24081},[615],[47,24083,24085],{"className":24084},[525],[47,24086,24088],{"className":24087,"style":17703},[529],[47,24089],{},[47,24091,1187],{"className":24092},[512],[47,24094,24096,24099],{"className":24095},[508],[47,24097,13164],{"className":24098},[508,581],[47,24100,24102],{"className":24101},[585],[47,24103,24105,24125],{"className":24104},[520,521],[47,24106,24108,24122],{"className":24107},[525],[47,24109,24111],{"className":24110,"style":13094},[529],[47,24112,24113,24116],{"style":1638},[47,24114],{"className":24115,"style":929},[537],[47,24117,24119],{"className":24118},[542,543,544,545],[47,24120,1166],{"className":24121},[508,581,545],[47,24123,616],{"className":24124},[615],[47,24126,24128],{"className":24127},[525],[47,24129,24131],{"className":24130,"style":1657},[529],[47,24132],{},[47,24134,1192],{"className":24135},[1299],[47,24137],{"className":24138,"style":1303},[888],[47,24140,285],{"className":24141},[508,581],[47,24143,1201],{"className":24144},[702],[47,24146],{"className":24147,"style":889},[888],[47,24149,82],{"className":24150},[893],[47,24152],{"className":24153,"style":889},[888],[47,24155,24157,24160,24200,24203,24206],{"className":24156},[499],[47,24158],{"className":24159,"style":22274},[503],[47,24161,24163,24166],{"className":24162},[508],[47,24164,8196],{"className":24165,"style":3580},[508,581],[47,24167,24169],{"className":24168},[585],[47,24170,24172,24192],{"className":24171},[520,521],[47,24173,24175,24189],{"className":24174},[525],[47,24176,24178],{"className":24177,"style":13094},[529],[47,24179,24180,24183],{"style":13467},[47,24181],{"className":24182,"style":929},[537],[47,24184,24186],{"className":24185},[542,543,544,545],[47,24187,1166],{"className":24188},[508,581,545],[47,24190,616],{"className":24191},[615],[47,24193,24195],{"className":24194},[525],[47,24196,24198],{"className":24197,"style":1657},[529],[47,24199],{},[47,24201],{"className":24202,"style":2259},[888],[47,24204,6147],{"className":24205},[942],[47,24207],{"className":24208,"style":2259},[888],[47,24210,24212,24215,24218,24267,24270,24273],{"className":24211},[499],[47,24213],{"className":24214,"style":22920},[503],[47,24216,19357],{"className":24217,"style":19604},[508,581],[47,24219,24221,24224],{"className":24220},[508],[47,24222,8196],{"className":24223,"style":3580},[508,581],[47,24225,24227],{"className":24226},[585],[47,24228,24230,24259],{"className":24229},[520,521],[47,24231,24233,24256],{"className":24232},[525],[47,24234,24236],{"className":24235,"style":3461},[529],[47,24237,24238,24241],{"style":13467},[47,24239],{"className":24240,"style":929},[537],[47,24242,24244],{"className":24243},[542,543,544,545],[47,24245,24247,24250,24253],{"className":24246},[508,545],[47,24248,1166],{"className":24249},[508,581,545],[47,24251,6147],{"className":24252},[942,545],[47,24254,472],{"className":24255},[508,545],[47,24257,616],{"className":24258},[615],[47,24260,24262],{"className":24261},[525],[47,24263,24265],{"className":24264,"style":7308},[529],[47,24266],{},[47,24268],{"className":24269,"style":2259},[888],[47,24271,6147],{"className":24272},[942],[47,24274],{"className":24275,"style":2259},[888],[47,24277,24279,24282,24311,24351,24354,24403,24406,24409,24412],{"className":24278},[499],[47,24280],{"className":24281,"style":23653},[503],[47,24283,24285,24288],{"className":24284},[508],[47,24286,19357],{"className":24287,"style":19604},[508,581],[47,24289,24291],{"className":24290},[585],[47,24292,24294],{"className":24293},[520],[47,24295,24297],{"className":24296},[525],[47,24298,24300],{"className":24299,"style":23672},[529],[47,24301,24302,24305],{"style":16788},[47,24303],{"className":24304,"style":929},[537],[47,24306,24308],{"className":24307},[542,543,544,545],[47,24309,3192],{"className":24310},[508,545],[47,24312,24314,24317],{"className":24313},[508],[47,24315,1061],{"className":24316,"style":1091},[508,581],[47,24318,24320],{"className":24319},[585],[47,24321,24323,24343],{"className":24322},[520,521],[47,24324,24326,24340],{"className":24325},[525],[47,24327,24329],{"className":24328,"style":1635},[529],[47,24330,24331,24334],{"style":20867},[47,24332],{"className":24333,"style":929},[537],[47,24335,24337],{"className":24336},[542,543,544,545],[47,24338,14413],{"className":24339,"style":3580},[508,581,545],[47,24341,616],{"className":24342},[615],[47,24344,24346],{"className":24345},[525],[47,24347,24349],{"className":24348,"style":1657},[529],[47,24350],{},[47,24352,1187],{"className":24353},[512],[47,24355,24357,24360],{"className":24356},[508],[47,24358,13164],{"className":24359},[508,581],[47,24361,24363],{"className":24362},[585],[47,24364,24366,24395],{"className":24365},[520,521],[47,24367,24369,24392],{"className":24368},[525],[47,24370,24372],{"className":24371,"style":3461},[529],[47,24373,24374,24377],{"style":1638},[47,24375],{"className":24376,"style":929},[537],[47,24378,24380],{"className":24379},[542,543,544,545],[47,24381,24383,24386,24389],{"className":24382},[508,545],[47,24384,1166],{"className":24385},[508,581,545],[47,24387,6147],{"className":24388},[942,545],[47,24390,3192],{"className":24391},[508,545],[47,24393,616],{"className":24394},[615],[47,24396,24398],{"className":24397},[525],[47,24399,24401],{"className":24400,"style":7308},[529],[47,24402],{},[47,24404,1201],{"className":24405},[702],[47,24407],{"className":24408,"style":2259},[888],[47,24410,18727],{"className":24411},[942],[47,24413],{"className":24414,"style":2259},[888],[47,24416,24418,24421,24461,24464,24504],{"className":24417},[499],[47,24419],{"className":24420,"style":1264},[503],[47,24422,24424,24427],{"className":24423},[508],[47,24425,1061],{"className":24426,"style":1091},[508,581],[47,24428,24430],{"className":24429},[585],[47,24431,24433,24453],{"className":24432},[520,521],[47,24434,24436,24450],{"className":24435},[525],[47,24437,24439],{"className":24438,"style":1635},[529],[47,24440,24441,24444],{"style":20867},[47,24442],{"className":24443,"style":929},[537],[47,24445,24447],{"className":24446},[542,543,544,545],[47,24448,14413],{"className":24449,"style":3580},[508,581,545],[47,24451,616],{"className":24452},[615],[47,24454,24456],{"className":24455},[525],[47,24457,24459],{"className":24458,"style":1657},[529],[47,24460],{},[47,24462,1187],{"className":24463},[512],[47,24465,24467,24470],{"className":24466},[508],[47,24468,13164],{"className":24469},[508,581],[47,24471,24473],{"className":24472},[585],[47,24474,24476,24496],{"className":24475},[520,521],[47,24477,24479,24493],{"className":24478},[525],[47,24480,24482],{"className":24481,"style":13094},[529],[47,24483,24484,24487],{"style":1638},[47,24485],{"className":24486,"style":929},[537],[47,24488,24490],{"className":24489},[542,543,544,545],[47,24491,1166],{"className":24492},[508,581,545],[47,24494,616],{"className":24495},[615],[47,24497,24499],{"className":24498},[525],[47,24500,24502],{"className":24501,"style":1657},[529],[47,24503],{},[47,24505,1201],{"className":24506},[702],[47,24508],{"className":24509},[888,22015],[47,24511,24513,24516,24567,24570,24610,24613,24616,24619,24622,24625,24628],{"className":24512},[499],[47,24514],{"className":24515,"style":23653},[503],[47,24517,24519,24522],{"className":24518},[508],[47,24520,1163],{"className":24521},[508,581],[47,24523,24525],{"className":24524},[585],[47,24526,24528,24559],{"className":24527},[520,521],[47,24529,24531,24556],{"className":24530},[525],[47,24532,24534,24545],{"className":24533,"style":23672},[529],[47,24535,24536,24539],{"style":17673},[47,24537],{"className":24538,"style":929},[537],[47,24540,24542],{"className":24541},[542,543,544,545],[47,24543,14413],{"className":24544,"style":3580},[508,581,545],[47,24546,24547,24550],{"style":16788},[47,24548],{"className":24549,"style":929},[537],[47,24551,24553],{"className":24552},[542,543,544,545],[47,24554,5041],{"className":24555},[508,545],[47,24557,616],{"className":24558},[615],[47,24560,24562],{"className":24561},[525],[47,24563,24565],{"className":24564,"style":17703},[529],[47,24566],{},[47,24568,1187],{"className":24569},[512],[47,24571,24573,24576],{"className":24572},[508],[47,24574,13164],{"className":24575},[508,581],[47,24577,24579],{"className":24578},[585],[47,24580,24582,24602],{"className":24581},[520,521],[47,24583,24585,24599],{"className":24584},[525],[47,24586,24588],{"className":24587,"style":13094},[529],[47,24589,24590,24593],{"style":1638},[47,24591],{"className":24592,"style":929},[537],[47,24594,24596],{"className":24595},[542,543,544,545],[47,24597,1166],{"className":24598},[508,581,545],[47,24600,616],{"className":24601},[615],[47,24603,24605],{"className":24604},[525],[47,24606,24608],{"className":24607,"style":1657},[529],[47,24609],{},[47,24611,1192],{"className":24612},[1299],[47,24614],{"className":24615,"style":1303},[888],[47,24617,285],{"className":24618},[508,581],[47,24620,1201],{"className":24621},[702],[47,24623],{"className":24624,"style":889},[888],[47,24626,82],{"className":24627},[893],[47,24629],{"className":24630,"style":889},[888],[47,24632,24634,24637,24677,24680,24683],{"className":24633},[499],[47,24635],{"className":24636,"style":22274},[503],[47,24638,24640,24643],{"className":24639},[508],[47,24641,8196],{"className":24642,"style":3580},[508,581],[47,24644,24646],{"className":24645},[585],[47,24647,24649,24669],{"className":24648},[520,521],[47,24650,24652,24666],{"className":24651},[525],[47,24653,24655],{"className":24654,"style":13094},[529],[47,24656,24657,24660],{"style":13467},[47,24658],{"className":24659,"style":929},[537],[47,24661,24663],{"className":24662},[542,543,544,545],[47,24664,1166],{"className":24665},[508,581,545],[47,24667,616],{"className":24668},[615],[47,24670,24672],{"className":24671},[525],[47,24673,24675],{"className":24674,"style":1657},[529],[47,24676],{},[47,24678],{"className":24679,"style":2259},[888],[47,24681,6147],{"className":24682},[942],[47,24684],{"className":24685,"style":2259},[888],[47,24687,24689,24693,24696,24699,24702],{"className":24688},[499],[47,24690],{"className":24691,"style":24692},[503],"height:0.6597em;vertical-align:-0.1944em;",[47,24694,19357],{"className":24695,"style":19604},[508,581],[47,24697],{"className":24698,"style":2259},[888],[47,24700,23411],{"className":24701},[942],[47,24703],{"className":24704,"style":2259},[888],[47,24706,24708,24711,24760,24763,24766],{"className":24707},[499],[47,24709],{"className":24710,"style":22920},[503],[47,24712,24714,24717],{"className":24713},[508],[47,24715,8196],{"className":24716,"style":3580},[508,581],[47,24718,24720],{"className":24719},[585],[47,24721,24723,24752],{"className":24722},[520,521],[47,24724,24726,24749],{"className":24725},[525],[47,24727,24729],{"className":24728,"style":3461},[529],[47,24730,24731,24734],{"style":13467},[47,24732],{"className":24733,"style":929},[537],[47,24735,24737],{"className":24736},[542,543,544,545],[47,24738,24740,24743,24746],{"className":24739},[508,545],[47,24741,1166],{"className":24742},[508,581,545],[47,24744,6147],{"className":24745},[942,545],[47,24747,472],{"className":24748},[508,545],[47,24750,616],{"className":24751},[615],[47,24753,24755],{"className":24754},[525],[47,24756,24758],{"className":24757,"style":7308},[529],[47,24759],{},[47,24761],{"className":24762,"style":2259},[888],[47,24764,6147],{"className":24765},[942],[47,24767],{"className":24768,"style":2259},[888],[47,24770,24772,24776,24805,24808,24811],{"className":24771},[499],[47,24773],{"className":24774,"style":24775},[503],"height:1.0585em;vertical-align:-0.1944em;",[47,24777,24779,24782],{"className":24778},[508],[47,24780,19357],{"className":24781,"style":19604},[508,581],[47,24783,24785],{"className":24784},[585],[47,24786,24788],{"className":24787},[520],[47,24789,24791],{"className":24790},[525],[47,24792,24794],{"className":24793,"style":23672},[529],[47,24795,24796,24799],{"style":16788},[47,24797],{"className":24798,"style":929},[537],[47,24800,24802],{"className":24801},[542,543,544,545],[47,24803,3192],{"className":24804},[508,545],[47,24806],{"className":24807,"style":2259},[888],[47,24809,23411],{"className":24810},[942],[47,24812],{"className":24813,"style":2259},[888],[47,24815,24817,24820,24869,24872,24875],{"className":24816},[499],[47,24818],{"className":24819,"style":22920},[503],[47,24821,24823,24826],{"className":24822},[508],[47,24824,8196],{"className":24825,"style":3580},[508,581],[47,24827,24829],{"className":24828},[585],[47,24830,24832,24861],{"className":24831},[520,521],[47,24833,24835,24858],{"className":24834},[525],[47,24836,24838],{"className":24837,"style":3461},[529],[47,24839,24840,24843],{"style":13467},[47,24841],{"className":24842,"style":929},[537],[47,24844,24846],{"className":24845},[542,543,544,545],[47,24847,24849,24852,24855],{"className":24848},[508,545],[47,24850,1166],{"className":24851},[508,581,545],[47,24853,6147],{"className":24854},[942,545],[47,24856,3192],{"className":24857},[508,545],[47,24859,616],{"className":24860},[615],[47,24862,24864],{"className":24863},[525],[47,24865,24867],{"className":24866,"style":7308},[529],[47,24868],{},[47,24870],{"className":24871,"style":2259},[888],[47,24873,6147],{"className":24874},[942],[47,24876],{"className":24877,"style":2259},[888],[47,24879,24881,24884,24913,24953,24956,25005,25008,25011,25014],{"className":24880},[499],[47,24882],{"className":24883,"style":23653},[503],[47,24885,24887,24890],{"className":24886},[508],[47,24888,19357],{"className":24889,"style":19604},[508,581],[47,24891,24893],{"className":24892},[585],[47,24894,24896],{"className":24895},[520],[47,24897,24899],{"className":24898},[525],[47,24900,24902],{"className":24901,"style":23672},[529],[47,24903,24904,24907],{"style":16788},[47,24905],{"className":24906,"style":929},[537],[47,24908,24910],{"className":24909},[542,543,544,545],[47,24911,5041],{"className":24912},[508,545],[47,24914,24916,24919],{"className":24915},[508],[47,24917,1061],{"className":24918,"style":1091},[508,581],[47,24920,24922],{"className":24921},[585],[47,24923,24925,24945],{"className":24924},[520,521],[47,24926,24928,24942],{"className":24927},[525],[47,24929,24931],{"className":24930,"style":1635},[529],[47,24932,24933,24936],{"style":20867},[47,24934],{"className":24935,"style":929},[537],[47,24937,24939],{"className":24938},[542,543,544,545],[47,24940,14413],{"className":24941,"style":3580},[508,581,545],[47,24943,616],{"className":24944},[615],[47,24946,24948],{"className":24947},[525],[47,24949,24951],{"className":24950,"style":1657},[529],[47,24952],{},[47,24954,1187],{"className":24955},[512],[47,24957,24959,24962],{"className":24958},[508],[47,24960,13164],{"className":24961},[508,581],[47,24963,24965],{"className":24964},[585],[47,24966,24968,24997],{"className":24967},[520,521],[47,24969,24971,24994],{"className":24970},[525],[47,24972,24974],{"className":24973,"style":3461},[529],[47,24975,24976,24979],{"style":1638},[47,24977],{"className":24978,"style":929},[537],[47,24980,24982],{"className":24981},[542,543,544,545],[47,24983,24985,24988,24991],{"className":24984},[508,545],[47,24986,1166],{"className":24987},[508,581,545],[47,24989,6147],{"className":24990},[942,545],[47,24992,5041],{"className":24993},[508,545],[47,24995,616],{"className":24996},[615],[47,24998,25000],{"className":24999},[525],[47,25001,25003],{"className":25002,"style":7308},[529],[47,25004],{},[47,25006,1201],{"className":25007},[702],[47,25009],{"className":25010,"style":2259},[888],[47,25012,18727],{"className":25013},[942],[47,25015],{"className":25016,"style":2259},[888],[47,25018,25020,25023,25063,25066,25106],{"className":25019},[499],[47,25021],{"className":25022,"style":1264},[503],[47,25024,25026,25029],{"className":25025},[508],[47,25027,1061],{"className":25028,"style":1091},[508,581],[47,25030,25032],{"className":25031},[585],[47,25033,25035,25055],{"className":25034},[520,521],[47,25036,25038,25052],{"className":25037},[525],[47,25039,25041],{"className":25040,"style":1635},[529],[47,25042,25043,25046],{"style":20867},[47,25044],{"className":25045,"style":929},[537],[47,25047,25049],{"className":25048},[542,543,544,545],[47,25050,14413],{"className":25051,"style":3580},[508,581,545],[47,25053,616],{"className":25054},[615],[47,25056,25058],{"className":25057},[525],[47,25059,25061],{"className":25060,"style":1657},[529],[47,25062],{},[47,25064,1187],{"className":25065},[512],[47,25067,25069,25072],{"className":25068},[508],[47,25070,13164],{"className":25071},[508,581],[47,25073,25075],{"className":25074},[585],[47,25076,25078,25098],{"className":25077},[520,521],[47,25079,25081,25095],{"className":25080},[525],[47,25082,25084],{"className":25083,"style":13094},[529],[47,25085,25086,25089],{"style":1638},[47,25087],{"className":25088,"style":929},[537],[47,25090,25092],{"className":25091},[542,543,544,545],[47,25093,1166],{"className":25094},[508,581,545],[47,25096,616],{"className":25097},[615],[47,25099,25101],{"className":25100},[525],[47,25102,25104],{"className":25103,"style":1657},[529],[47,25105],{},[47,25107,1201],{"className":25108},[702],[47,25110],{"className":25111},[888,22015],[47,25113,25115,25119],{"className":25114},[499],[47,25116],{"className":25117,"style":25118},[503],"height:1.53em;vertical-align:-0.03em;",[47,25120,25122,25125],{"className":25121},[508],[47,25123,23500],{"className":25124},[508],[47,25126],{"className":25127,"style":25129},[508,25128],"rule","border-right-width:0em;border-top-width:1.5em;bottom:0em;",[47,25131],{"className":25132},[888,22015],[47,25134,25136,25140,25192,25195,25235,25238,25241,25244,25247,25250,25253],{"className":25135},[499],[47,25137],{"className":25138,"style":25139},[503],"height:1.1413em;vertical-align:-0.25em;",[47,25141,25143,25146],{"className":25142},[508],[47,25144,1163],{"className":25145},[508,581],[47,25147,25149],{"className":25148},[585],[47,25150,25152,25184],{"className":25151},[520,521],[47,25153,25155,25181],{"className":25154},[525],[47,25156,25159,25170],{"className":25157,"style":25158},[529],"height:0.8913em;",[47,25160,25161,25164],{"style":17673},[47,25162],{"className":25163,"style":929},[537],[47,25165,25167],{"className":25166},[542,543,544,545],[47,25168,14413],{"className":25169,"style":3580},[508,581,545],[47,25171,25172,25175],{"style":16788},[47,25173],{"className":25174,"style":929},[537],[47,25176,25178],{"className":25177},[542,543,544,545],[47,25179,13604],{"className":25180,"style":3555},[508,581,545],[47,25182,616],{"className":25183},[615],[47,25185,25187],{"className":25186},[525],[47,25188,25190],{"className":25189,"style":17703},[529],[47,25191],{},[47,25193,1187],{"className":25194},[512],[47,25196,25198,25201],{"className":25197},[508],[47,25199,13164],{"className":25200},[508,581],[47,25202,25204],{"className":25203},[585],[47,25205,25207,25227],{"className":25206},[520,521],[47,25208,25210,25224],{"className":25209},[525],[47,25211,25213],{"className":25212,"style":13094},[529],[47,25214,25215,25218],{"style":1638},[47,25216],{"className":25217,"style":929},[537],[47,25219,25221],{"className":25220},[542,543,544,545],[47,25222,1166],{"className":25223},[508,581,545],[47,25225,616],{"className":25226},[615],[47,25228,25230],{"className":25229},[525],[47,25231,25233],{"className":25232,"style":1657},[529],[47,25234],{},[47,25236,1192],{"className":25237},[1299],[47,25239],{"className":25240,"style":1303},[888],[47,25242,285],{"className":25243},[508,581],[47,25245,1201],{"className":25246},[702],[47,25248],{"className":25249,"style":889},[888],[47,25251,82],{"className":25252},[893],[47,25254],{"className":25255,"style":889},[888],[47,25257,25259,25262,25302,25305,25308],{"className":25258},[499],[47,25260],{"className":25261,"style":22274},[503],[47,25263,25265,25268],{"className":25264},[508],[47,25266,8196],{"className":25267,"style":3580},[508,581],[47,25269,25271],{"className":25270},[585],[47,25272,25274,25294],{"className":25273},[520,521],[47,25275,25277,25291],{"className":25276},[525],[47,25278,25280],{"className":25279,"style":13094},[529],[47,25281,25282,25285],{"style":13467},[47,25283],{"className":25284,"style":929},[537],[47,25286,25288],{"className":25287},[542,543,544,545],[47,25289,1166],{"className":25290},[508,581,545],[47,25292,616],{"className":25293},[615],[47,25295,25297],{"className":25296},[525],[47,25298,25300],{"className":25299,"style":1657},[529],[47,25301],{},[47,25303],{"className":25304,"style":2259},[888],[47,25306,6147],{"className":25307},[942],[47,25309],{"className":25310,"style":2259},[888],[47,25312,25314,25317,25320,25323,25326],{"className":25313},[499],[47,25315],{"className":25316,"style":24692},[503],[47,25318,19357],{"className":25319,"style":19604},[508,581],[47,25321],{"className":25322,"style":2259},[888],[47,25324,23411],{"className":25325},[942],[47,25327],{"className":25328,"style":2259},[888],[47,25330,25332,25335,25384,25387,25390],{"className":25331},[499],[47,25333],{"className":25334,"style":22920},[503],[47,25336,25338,25341],{"className":25337},[508],[47,25339,8196],{"className":25340,"style":3580},[508,581],[47,25342,25344],{"className":25343},[585],[47,25345,25347,25376],{"className":25346},[520,521],[47,25348,25350,25373],{"className":25349},[525],[47,25351,25353],{"className":25352,"style":3461},[529],[47,25354,25355,25358],{"style":13467},[47,25356],{"className":25357,"style":929},[537],[47,25359,25361],{"className":25360},[542,543,544,545],[47,25362,25364,25367,25370],{"className":25363},[508,545],[47,25365,1166],{"className":25366},[508,581,545],[47,25368,6147],{"className":25369},[942,545],[47,25371,472],{"className":25372},[508,545],[47,25374,616],{"className":25375},[615],[47,25377,25379],{"className":25378},[525],[47,25380,25382],{"className":25381,"style":7308},[529],[47,25383],{},[47,25385],{"className":25386,"style":2259},[888],[47,25388,6147],{"className":25389},[942],[47,25391],{"className":25392,"style":2259},[888],[47,25394,25396,25399,25428,25431,25434],{"className":25395},[499],[47,25397],{"className":25398,"style":24775},[503],[47,25400,25402,25405],{"className":25401},[508],[47,25403,19357],{"className":25404,"style":19604},[508,581],[47,25406,25408],{"className":25407},[585],[47,25409,25411],{"className":25410},[520],[47,25412,25414],{"className":25413},[525],[47,25415,25417],{"className":25416,"style":23672},[529],[47,25418,25419,25422],{"style":16788},[47,25420],{"className":25421,"style":929},[537],[47,25423,25425],{"className":25424},[542,543,544,545],[47,25426,3192],{"className":25427},[508,545],[47,25429],{"className":25430,"style":2259},[888],[47,25432,23411],{"className":25433},[942],[47,25435],{"className":25436,"style":2259},[888],[47,25438,25440,25443,25492,25495,25498],{"className":25439},[499],[47,25441],{"className":25442,"style":22920},[503],[47,25444,25446,25449],{"className":25445},[508],[47,25447,8196],{"className":25448,"style":3580},[508,581],[47,25450,25452],{"className":25451},[585],[47,25453,25455,25484],{"className":25454},[520,521],[47,25456,25458,25481],{"className":25457},[525],[47,25459,25461],{"className":25460,"style":3461},[529],[47,25462,25463,25466],{"style":13467},[47,25464],{"className":25465,"style":929},[537],[47,25467,25469],{"className":25468},[542,543,544,545],[47,25470,25472,25475,25478],{"className":25471},[508,545],[47,25473,1166],{"className":25474},[508,581,545],[47,25476,6147],{"className":25477},[942,545],[47,25479,3192],{"className":25480},[508,545],[47,25482,616],{"className":25483},[615],[47,25485,25487],{"className":25486},[525],[47,25488,25490],{"className":25489,"style":7308},[529],[47,25491],{},[47,25493],{"className":25494,"style":2259},[888],[47,25496,6147],{"className":25497},[942],[47,25499],{"className":25500,"style":2259},[888],[47,25502,25504,25507,25536,25539,25542],{"className":25503},[499],[47,25505],{"className":25506,"style":24775},[503],[47,25508,25510,25513],{"className":25509},[508],[47,25511,19357],{"className":25512,"style":19604},[508,581],[47,25514,25516],{"className":25515},[585],[47,25517,25519],{"className":25518},[520],[47,25520,25522],{"className":25521},[525],[47,25523,25525],{"className":25524,"style":23672},[529],[47,25526,25527,25530],{"style":16788},[47,25528],{"className":25529,"style":929},[537],[47,25531,25533],{"className":25532},[542,543,544,545],[47,25534,5041],{"className":25535},[508,545],[47,25537],{"className":25538,"style":2259},[888],[47,25540,23411],{"className":25541},[942],[47,25543],{"className":25544,"style":2259},[888],[47,25546,25548,25551,25600,25603,25606],{"className":25547},[499],[47,25549],{"className":25550,"style":22920},[503],[47,25552,25554,25557],{"className":25553},[508],[47,25555,8196],{"className":25556,"style":3580},[508,581],[47,25558,25560],{"className":25559},[585],[47,25561,25563,25592],{"className":25562},[520,521],[47,25564,25566,25589],{"className":25565},[525],[47,25567,25569],{"className":25568,"style":3461},[529],[47,25570,25571,25574],{"style":13467},[47,25572],{"className":25573,"style":929},[537],[47,25575,25577],{"className":25576},[542,543,544,545],[47,25578,25580,25583,25586],{"className":25579},[508,545],[47,25581,1166],{"className":25582},[508,581,545],[47,25584,6147],{"className":25585},[942,545],[47,25587,5041],{"className":25588},[508,545],[47,25590,616],{"className":25591},[615],[47,25593,25595],{"className":25594},[525],[47,25596,25598],{"className":25597,"style":7308},[529],[47,25599],{},[47,25601],{"className":25602,"style":2259},[888],[47,25604,6147],{"className":25605},[942],[47,25607],{"className":25608,"style":2259},[888],[47,25610,25612,25616,25619,25622,25625],{"className":25611},[499],[47,25613],{"className":25614,"style":25615},[503],"height:0.6667em;vertical-align:-0.0833em;",[47,25617,23606],{"className":25618},[1349],[47,25620],{"className":25621,"style":2259},[888],[47,25623,6147],{"className":25624},[942],[47,25626],{"className":25627,"style":2259},[888],[47,25629,25631,25635,25664,25667,25670],{"className":25630},[499],[47,25632],{"className":25633,"style":25634},[503],"height:1.0858em;vertical-align:-0.1944em;",[47,25636,25638,25641],{"className":25637},[508],[47,25639,19357],{"className":25640,"style":19604},[508,581],[47,25642,25644],{"className":25643},[585],[47,25645,25647],{"className":25646},[520],[47,25648,25650],{"className":25649},[525],[47,25651,25653],{"className":25652,"style":25158},[529],[47,25654,25655,25658],{"style":16788},[47,25656],{"className":25657,"style":929},[537],[47,25659,25661],{"className":25660},[542,543,544,545],[47,25662,13604],{"className":25663,"style":3555},[508,581,545],[47,25665],{"className":25666,"style":2259},[888],[47,25668,23411],{"className":25669},[942],[47,25671],{"className":25672,"style":2259},[888],[47,25674,25676,25679,25719,25722,25725],{"className":25675},[499],[47,25677],{"className":25678,"style":22274},[503],[47,25680,25682,25685],{"className":25681},[508],[47,25683,8196],{"className":25684,"style":3580},[508,581],[47,25686,25688],{"className":25687},[585],[47,25689,25691,25711],{"className":25690},[520,521],[47,25692,25694,25708],{"className":25693},[525],[47,25695,25697],{"className":25696,"style":13827},[529],[47,25698,25699,25702],{"style":13467},[47,25700],{"className":25701,"style":929},[537],[47,25703,25705],{"className":25704},[542,543,544,545],[47,25706,13604],{"className":25707,"style":3555},[508,581,545],[47,25709,616],{"className":25710},[615],[47,25712,25714],{"className":25713},[525],[47,25715,25717],{"className":25716,"style":1657},[529],[47,25718],{},[47,25720],{"className":25721,"style":2259},[888],[47,25723,18727],{"className":25724},[942],[47,25726],{"className":25727,"style":2259},[888],[47,25729,25731,25734,25774,25777,25817],{"className":25730},[499],[47,25732],{"className":25733,"style":1264},[503],[47,25735,25737,25740],{"className":25736},[508],[47,25738,1061],{"className":25739,"style":1091},[508,581],[47,25741,25743],{"className":25742},[585],[47,25744,25746,25766],{"className":25745},[520,521],[47,25747,25749,25763],{"className":25748},[525],[47,25750,25752],{"className":25751,"style":1635},[529],[47,25753,25754,25757],{"style":20867},[47,25755],{"className":25756,"style":929},[537],[47,25758,25760],{"className":25759},[542,543,544,545],[47,25761,14413],{"className":25762,"style":3580},[508,581,545],[47,25764,616],{"className":25765},[615],[47,25767,25769],{"className":25768},[525],[47,25770,25772],{"className":25771,"style":1657},[529],[47,25773],{},[47,25775,1187],{"className":25776},[512],[47,25778,25780,25783],{"className":25779},[508],[47,25781,13164],{"className":25782},[508,581],[47,25784,25786],{"className":25785},[585],[47,25787,25789,25809],{"className":25788},[520,521],[47,25790,25792,25806],{"className":25791},[525],[47,25793,25795],{"className":25794,"style":13094},[529],[47,25796,25797,25800],{"style":1638},[47,25798],{"className":25799,"style":929},[537],[47,25801,25803],{"className":25802},[542,543,544,545],[47,25804,1166],{"className":25805},[508,581,545],[47,25807,616],{"className":25808},[615],[47,25810,25812],{"className":25811},[525],[47,25813,25815],{"className":25814,"style":1657},[529],[47,25816],{},[47,25818,1201],{"className":25819},[702],[14,25821,25822,25823,25896,25897,25925],{},"为了让式子表示整洁，我们定义一个中间变量，令 ",[47,25824,25826,25845],{"className":25825},[451],[47,25827,25829],{"className":25828},[455],[457,25830,25831],{"xmlns":459},[461,25832,25833,25842],{},[464,25834,25835],{},[477,25836,25837,25840],{},[480,25838,25839],{},"δ",[480,25841,1166],{},[487,25843,25844],{"encoding":489},"\\delta_t",[47,25846,25848],{"className":25847,"ariaHidden":495},[494],[47,25849,25851,25854],{"className":25850},[499],[47,25852],{"className":25853,"style":1616},[503],[47,25855,25857,25861],{"className":25856},[508],[47,25858,25839],{"className":25859,"style":25860},[508,581],"margin-right:0.03785em;",[47,25862,25864],{"className":25863},[585],[47,25865,25867,25888],{"className":25866},[520,521],[47,25868,25870,25885],{"className":25869},[525],[47,25871,25873],{"className":25872,"style":13094},[529],[47,25874,25876,25879],{"style":25875},"top:-2.55em;margin-left:-0.0379em;margin-right:0.05em;",[47,25877],{"className":25878,"style":929},[537],[47,25880,25882],{"className":25881},[542,543,544,545],[47,25883,1166],{"className":25884},[508,581,545],[47,25886,616],{"className":25887},[615],[47,25889,25891],{"className":25890},[525],[47,25892,25894],{"className":25893,"style":1657},[529],[47,25895],{}," 表示在第 ",[47,25898,25900,25913],{"className":25899},[451],[47,25901,25903],{"className":25902},[455],[457,25904,25905],{"xmlns":459},[461,25906,25907,25911],{},[464,25908,25909],{},[480,25910,1166],{},[487,25912,1166],{"encoding":489},[47,25914,25916],{"className":25915,"ariaHidden":495},[494],[47,25917,25919,25922],{"className":25918},[499],[47,25920],{"className":25921,"style":13143},[503],[47,25923,1166],{"className":25924},[508,581]," 步采取特定动作带来的优势。",[47,25927,25929],{"className":25928},[14712],[47,25930,25932,26258],{"className":25931},[451],[47,25933,25935],{"className":25934},[455],[457,25936,25937],{"xmlns":459,"display":9848},[461,25938,25939,26255],{},[464,25940,25941,25949,25951,25957,25959,25961,25963,25969,25971,25983,25985,25987,25993,25995,26001,26003,26005,26019,26021,26033,26035,26037,26039,26045,26047,26059,26061,26063,26069,26071,26083,26085,26087,26095,26097,26103,26105,26107,26109,26111,26119,26121,26129,26131,26137,26139,26141,26143,26145,26153,26155,26157,26171,26173,26181,26183,26189,26191,26193,26195,26197,26205,26207,26209,26223,26225,26231,26245,26247],{},[2209,25942,25943,25945,25947],{},[480,25944,25839],{},[480,25946,1166],{},[480,25948,1061],{},[850,25950,82],{},[477,25952,25953,25955],{},[480,25954,8196],{},[480,25956,1166],{},[850,25958,6147],{},[480,25960,19357],{},[850,25962,23411],{},[477,25964,25965,25967],{},[480,25966,1061],{},[480,25968,14413],{},[850,25970,1187],{"stretchy":1186},[477,25972,25973,25975],{},[480,25974,13164],{},[464,25976,25977,25979,25981],{},[480,25978,1166],{},[850,25980,6147],{},[470,25982,472],{},[850,25984,1201],{"stretchy":1186},[850,25986,18727],{},[477,25988,25989,25991],{},[480,25990,1061],{},[480,25992,14413],{},[850,25994,1187],{"stretchy":1186},[477,25996,25997,25999],{},[480,25998,13164],{},[480,26000,1166],{},[850,26002,1201],{"stretchy":1186},[888,26004],{"linebreak":22015},[2209,26006,26007,26009,26017],{},[480,26008,25839],{},[464,26010,26011,26013,26015],{},[480,26012,1166],{},[850,26014,6147],{},[470,26016,472],{},[480,26018,1061],{},[850,26020,82],{},[477,26022,26023,26025],{},[480,26024,8196],{},[464,26026,26027,26029,26031],{},[480,26028,1166],{},[850,26030,6147],{},[470,26032,472],{},[850,26034,6147],{},[480,26036,19357],{},[850,26038,23411],{},[477,26040,26041,26043],{},[480,26042,1061],{},[480,26044,14413],{},[850,26046,1187],{"stretchy":1186},[477,26048,26049,26051],{},[480,26050,13164],{},[464,26052,26053,26055,26057],{},[480,26054,1166],{},[850,26056,6147],{},[470,26058,3192],{},[850,26060,1201],{"stretchy":1186},[850,26062,18727],{},[477,26064,26065,26067],{},[480,26066,1061],{},[480,26068,14413],{},[850,26070,1187],{"stretchy":1186},[477,26072,26073,26075],{},[480,26074,13164],{},[464,26076,26077,26079,26081],{},[480,26078,1166],{},[850,26080,6147],{},[470,26082,472],{},[850,26084,1201],{"stretchy":1186},[888,26086],{"linebreak":22015},[2209,26088,26089,26091,26093],{},[480,26090,1163],{},[480,26092,14413],{},[470,26094,472],{},[850,26096,1187],{"stretchy":1186},[477,26098,26099,26101],{},[480,26100,13164],{},[480,26102,1166],{},[850,26104,1192],{"separator":495},[480,26106,285],{},[850,26108,1201],{"stretchy":1186},[850,26110,82],{},[2209,26112,26113,26115,26117],{},[480,26114,25839],{},[480,26116,1166],{},[480,26118,1061],{},[888,26120],{"linebreak":22015},[2209,26122,26123,26125,26127],{},[480,26124,1163],{},[480,26126,14413],{},[470,26128,3192],{},[850,26130,1187],{"stretchy":1186},[477,26132,26133,26135],{},[480,26134,13164],{},[480,26136,1166],{},[850,26138,1192],{"separator":495},[480,26140,285],{},[850,26142,1201],{"stretchy":1186},[850,26144,82],{},[2209,26146,26147,26149,26151],{},[480,26148,25839],{},[480,26150,1166],{},[480,26152,1061],{},[850,26154,6147],{},[480,26156,19357],{},[2209,26158,26159,26161,26169],{},[480,26160,25839],{},[464,26162,26163,26165,26167],{},[480,26164,1166],{},[850,26166,6147],{},[470,26168,472],{},[480,26170,1061],{},[888,26172],{"linebreak":22015},[2209,26174,26175,26177,26179],{},[480,26176,1163],{},[480,26178,14413],{},[470,26180,5041],{},[850,26182,1187],{"stretchy":1186},[477,26184,26185,26187],{},[480,26186,13164],{},[480,26188,1166],{},[850,26190,1192],{"separator":495},[480,26192,285],{},[850,26194,1201],{"stretchy":1186},[850,26196,82],{},[2209,26198,26199,26201,26203],{},[480,26200,25839],{},[480,26202,1166],{},[480,26204,1061],{},[850,26206,6147],{},[480,26208,19357],{},[2209,26210,26211,26213,26221],{},[480,26212,25839],{},[464,26214,26215,26217,26219],{},[480,26216,1166],{},[850,26218,6147],{},[470,26220,472],{},[480,26222,1061],{},[850,26224,6147],{},[854,26226,26227,26229],{},[480,26228,19357],{},[470,26230,3192],{},[2209,26232,26233,26235,26243],{},[480,26234,25839],{},[464,26236,26237,26239,26241],{},[480,26238,1166],{},[850,26240,6147],{},[470,26242,3192],{},[480,26244,1061],{},[888,26246],{"linebreak":22015},[464,26248,26249,26251],{},[480,26250,23500],{"mathvariant":1206},[23502,26252,26253],{"height":14728,"voffset":14728},[888,26254],{"mathbackground":23506,"width":14728,"height":23507},[487,26256,26257],{"encoding":489},"\n\\delta_t^V = r_t + \\gamma * V_\\theta(s_{t+1}) - V_\\theta(s_t)\\\\\n\n\\delta_{t+1}^V = r_{t+1} + \\gamma * V_\\theta(s_{t+2}) - V_\\theta(s_{t+1})\\\\\n\nA_\\theta^1(s_t, a) = \\delta_t^V\\\\\n\nA_\\theta^2(s_t, a) = \\delta_t^V + \\gamma \\delta_{t+1}^V\\\\\n\nA_\\theta^3(s_t, a) = \\delta_t^V + \\gamma \\delta_{t+1}^V + \\gamma^2 \\delta_{t+2}^V\\\\\n\n\\vdots",[47,26259,26261,26329,26384,26402,26512,26604,26607,26684,26748,26766,26876,26977,26980,27101,27158,27161,27282,27348,27417,27420,27541,27607,27685,27780,27783],{"className":26260,"ariaHidden":495},[494],[47,26262,26264,26268,26320,26323,26326],{"className":26263},[499],[47,26265],{"className":26266,"style":26267},[503],"height:1.1383em;vertical-align:-0.247em;",[47,26269,26271,26274],{"className":26270},[508],[47,26272,25839],{"className":26273,"style":25860},[508,581],[47,26275,26277],{"className":26276},[585],[47,26278,26280,26312],{"className":26279},[520,521],[47,26281,26283,26309],{"className":26282},[525],[47,26284,26286,26298],{"className":26285,"style":25158},[529],[47,26287,26289,26292],{"style":26288},"top:-2.453em;margin-left:-0.0379em;margin-right:0.05em;",[47,26290],{"className":26291,"style":929},[537],[47,26293,26295],{"className":26294},[542,543,544,545],[47,26296,1166],{"className":26297},[508,581,545],[47,26299,26300,26303],{"style":16788},[47,26301],{"className":26302,"style":929},[537],[47,26304,26306],{"className":26305},[542,543,544,545],[47,26307,1061],{"className":26308,"style":1091},[508,581,545],[47,26310,616],{"className":26311},[615],[47,26313,26315],{"className":26314},[525],[47,26316,26318],{"className":26317,"style":17703},[529],[47,26319],{},[47,26321],{"className":26322,"style":889},[888],[47,26324,82],{"className":26325},[893],[47,26327],{"className":26328,"style":889},[888],[47,26330,26332,26335,26375,26378,26381],{"className":26331},[499],[47,26333],{"className":26334,"style":22274},[503],[47,26336,26338,26341],{"className":26337},[508],[47,26339,8196],{"className":26340,"style":3580},[508,581],[47,26342,26344],{"className":26343},[585],[47,26345,26347,26367],{"className":26346},[520,521],[47,26348,26350,26364],{"className":26349},[525],[47,26351,26353],{"className":26352,"style":13094},[529],[47,26354,26355,26358],{"style":13467},[47,26356],{"className":26357,"style":929},[537],[47,26359,26361],{"className":26360},[542,543,544,545],[47,26362,1166],{"className":26363},[508,581,545],[47,26365,616],{"className":26366},[615],[47,26368,26370],{"className":26369},[525],[47,26371,26373],{"className":26372,"style":1657},[529],[47,26374],{},[47,26376],{"className":26377,"style":2259},[888],[47,26379,6147],{"className":26380},[942],[47,26382],{"className":26383,"style":2259},[888],[47,26385,26387,26390,26393,26396,26399],{"className":26386},[499],[47,26388],{"className":26389,"style":24692},[503],[47,26391,19357],{"className":26392,"style":19604},[508,581],[47,26394],{"className":26395,"style":2259},[888],[47,26397,23411],{"className":26398},[942],[47,26400],{"className":26401,"style":2259},[888],[47,26403,26405,26408,26448,26451,26500,26503,26506,26509],{"className":26404},[499],[47,26406],{"className":26407,"style":1264},[503],[47,26409,26411,26414],{"className":26410},[508],[47,26412,1061],{"className":26413,"style":1091},[508,581],[47,26415,26417],{"className":26416},[585],[47,26418,26420,26440],{"className":26419},[520,521],[47,26421,26423,26437],{"className":26422},[525],[47,26424,26426],{"className":26425,"style":1635},[529],[47,26427,26428,26431],{"style":20867},[47,26429],{"className":26430,"style":929},[537],[47,26432,26434],{"className":26433},[542,543,544,545],[47,26435,14413],{"className":26436,"style":3580},[508,581,545],[47,26438,616],{"className":26439},[615],[47,26441,26443],{"className":26442},[525],[47,26444,26446],{"className":26445,"style":1657},[529],[47,26447],{},[47,26449,1187],{"className":26450},[512],[47,26452,26454,26457],{"className":26453},[508],[47,26455,13164],{"className":26456},[508,581],[47,26458,26460],{"className":26459},[585],[47,26461,26463,26492],{"className":26462},[520,521],[47,26464,26466,26489],{"className":26465},[525],[47,26467,26469],{"className":26468,"style":3461},[529],[47,26470,26471,26474],{"style":1638},[47,26472],{"className":26473,"style":929},[537],[47,26475,26477],{"className":26476},[542,543,544,545],[47,26478,26480,26483,26486],{"className":26479},[508,545],[47,26481,1166],{"className":26482},[508,581,545],[47,26484,6147],{"className":26485},[942,545],[47,26487,472],{"className":26488},[508,545],[47,26490,616],{"className":26491},[615],[47,26493,26495],{"className":26494},[525],[47,26496,26498],{"className":26497,"style":7308},[529],[47,26499],{},[47,26501,1201],{"className":26502},[702],[47,26504],{"className":26505,"style":2259},[888],[47,26507,18727],{"className":26508},[942],[47,26510],{"className":26511,"style":2259},[888],[47,26513,26515,26518,26558,26561,26601],{"className":26514},[499],[47,26516],{"className":26517,"style":1264},[503],[47,26519,26521,26524],{"className":26520},[508],[47,26522,1061],{"className":26523,"style":1091},[508,581],[47,26525,26527],{"className":26526},[585],[47,26528,26530,26550],{"className":26529},[520,521],[47,26531,26533,26547],{"className":26532},[525],[47,26534,26536],{"className":26535,"style":1635},[529],[47,26537,26538,26541],{"style":20867},[47,26539],{"className":26540,"style":929},[537],[47,26542,26544],{"className":26543},[542,543,544,545],[47,26545,14413],{"className":26546,"style":3580},[508,581,545],[47,26548,616],{"className":26549},[615],[47,26551,26553],{"className":26552},[525],[47,26554,26556],{"className":26555,"style":1657},[529],[47,26557],{},[47,26559,1187],{"className":26560},[512],[47,26562,26564,26567],{"className":26563},[508],[47,26565,13164],{"className":26566},[508,581],[47,26568,26570],{"className":26569},[585],[47,26571,26573,26593],{"className":26572},[520,521],[47,26574,26576,26590],{"className":26575},[525],[47,26577,26579],{"className":26578,"style":13094},[529],[47,26580,26581,26584],{"style":1638},[47,26582],{"className":26583,"style":929},[537],[47,26585,26587],{"className":26586},[542,543,544,545],[47,26588,1166],{"className":26589},[508,581,545],[47,26591,616],{"className":26592},[615],[47,26594,26596],{"className":26595},[525],[47,26597,26599],{"className":26598,"style":1657},[529],[47,26600],{},[47,26602,1201],{"className":26603},[702],[47,26605],{"className":26606},[888,22015],[47,26608,26610,26614,26675,26678,26681],{"className":26609},[499],[47,26611],{"className":26612,"style":26613},[503],"height:1.1967em;vertical-align:-0.3053em;",[47,26615,26617,26620],{"className":26616},[508],[47,26618,25839],{"className":26619,"style":25860},[508,581],[47,26621,26623],{"className":26622},[585],[47,26624,26626,26666],{"className":26625},[520,521],[47,26627,26629,26663],{"className":26628},[525],[47,26630,26632,26652],{"className":26631,"style":25158},[529],[47,26633,26634,26637],{"style":26288},[47,26635],{"className":26636,"style":929},[537],[47,26638,26640],{"className":26639},[542,543,544,545],[47,26641,26643,26646,26649],{"className":26642},[508,545],[47,26644,1166],{"className":26645},[508,581,545],[47,26647,6147],{"className":26648},[942,545],[47,26650,472],{"className":26651},[508,545],[47,26653,26654,26657],{"style":16788},[47,26655],{"className":26656,"style":929},[537],[47,26658,26660],{"className":26659},[542,543,544,545],[47,26661,1061],{"className":26662,"style":1091},[508,581,545],[47,26664,616],{"className":26665},[615],[47,26667,26669],{"className":26668},[525],[47,26670,26673],{"className":26671,"style":26672},[529],"height:0.3053em;",[47,26674],{},[47,26676],{"className":26677,"style":889},[888],[47,26679,82],{"className":26680},[893],[47,26682],{"className":26683,"style":889},[888],[47,26685,26687,26690,26739,26742,26745],{"className":26686},[499],[47,26688],{"className":26689,"style":22920},[503],[47,26691,26693,26696],{"className":26692},[508],[47,26694,8196],{"className":26695,"style":3580},[508,581],[47,26697,26699],{"className":26698},[585],[47,26700,26702,26731],{"className":26701},[520,521],[47,26703,26705,26728],{"className":26704},[525],[47,26706,26708],{"className":26707,"style":3461},[529],[47,26709,26710,26713],{"style":13467},[47,26711],{"className":26712,"style":929},[537],[47,26714,26716],{"className":26715},[542,543,544,545],[47,26717,26719,26722,26725],{"className":26718},[508,545],[47,26720,1166],{"className":26721},[508,581,545],[47,26723,6147],{"className":26724},[942,545],[47,26726,472],{"className":26727},[508,545],[47,26729,616],{"className":26730},[615],[47,26732,26734],{"className":26733},[525],[47,26735,26737],{"className":26736,"style":7308},[529],[47,26738],{},[47,26740],{"className":26741,"style":2259},[888],[47,26743,6147],{"className":26744},[942],[47,26746],{"className":26747,"style":2259},[888],[47,26749,26751,26754,26757,26760,26763],{"className":26750},[499],[47,26752],{"className":26753,"style":24692},[503],[47,26755,19357],{"className":26756,"style":19604},[508,581],[47,26758],{"className":26759,"style":2259},[888],[47,26761,23411],{"className":26762},[942],[47,26764],{"className":26765,"style":2259},[888],[47,26767,26769,26772,26812,26815,26864,26867,26870,26873],{"className":26768},[499],[47,26770],{"className":26771,"style":1264},[503],[47,26773,26775,26778],{"className":26774},[508],[47,26776,1061],{"className":26777,"style":1091},[508,581],[47,26779,26781],{"className":26780},[585],[47,26782,26784,26804],{"className":26783},[520,521],[47,26785,26787,26801],{"className":26786},[525],[47,26788,26790],{"className":26789,"style":1635},[529],[47,26791,26792,26795],{"style":20867},[47,26793],{"className":26794,"style":929},[537],[47,26796,26798],{"className":26797},[542,543,544,545],[47,26799,14413],{"className":26800,"style":3580},[508,581,545],[47,26802,616],{"className":26803},[615],[47,26805,26807],{"className":26806},[525],[47,26808,26810],{"className":26809,"style":1657},[529],[47,26811],{},[47,26813,1187],{"className":26814},[512],[47,26816,26818,26821],{"className":26817},[508],[47,26819,13164],{"className":26820},[508,581],[47,26822,26824],{"className":26823},[585],[47,26825,26827,26856],{"className":26826},[520,521],[47,26828,26830,26853],{"className":26829},[525],[47,26831,26833],{"className":26832,"style":3461},[529],[47,26834,26835,26838],{"style":1638},[47,26836],{"className":26837,"style":929},[537],[47,26839,26841],{"className":26840},[542,543,544,545],[47,26842,26844,26847,26850],{"className":26843},[508,545],[47,26845,1166],{"className":26846},[508,581,545],[47,26848,6147],{"className":26849},[942,545],[47,26851,3192],{"className":26852},[508,545],[47,26854,616],{"className":26855},[615],[47,26857,26859],{"className":26858},[525],[47,26860,26862],{"className":26861,"style":7308},[529],[47,26863],{},[47,26865,1201],{"className":26866},[702],[47,26868],{"className":26869,"style":2259},[888],[47,26871,18727],{"className":26872},[942],[47,26874],{"className":26875,"style":2259},[888],[47,26877,26879,26882,26922,26925,26974],{"className":26878},[499],[47,26880],{"className":26881,"style":1264},[503],[47,26883,26885,26888],{"className":26884},[508],[47,26886,1061],{"className":26887,"style":1091},[508,581],[47,26889,26891],{"className":26890},[585],[47,26892,26894,26914],{"className":26893},[520,521],[47,26895,26897,26911],{"className":26896},[525],[47,26898,26900],{"className":26899,"style":1635},[529],[47,26901,26902,26905],{"style":20867},[47,26903],{"className":26904,"style":929},[537],[47,26906,26908],{"className":26907},[542,543,544,545],[47,26909,14413],{"className":26910,"style":3580},[508,581,545],[47,26912,616],{"className":26913},[615],[47,26915,26917],{"className":26916},[525],[47,26918,26920],{"className":26919,"style":1657},[529],[47,26921],{},[47,26923,1187],{"className":26924},[512],[47,26926,26928,26931],{"className":26927},[508],[47,26929,13164],{"className":26930},[508,581],[47,26932,26934],{"className":26933},[585],[47,26935,26937,26966],{"className":26936},[520,521],[47,26938,26940,26963],{"className":26939},[525],[47,26941,26943],{"className":26942,"style":3461},[529],[47,26944,26945,26948],{"style":1638},[47,26946],{"className":26947,"style":929},[537],[47,26949,26951],{"className":26950},[542,543,544,545],[47,26952,26954,26957,26960],{"className":26953},[508,545],[47,26955,1166],{"className":26956},[508,581,545],[47,26958,6147],{"className":26959},[942,545],[47,26961,472],{"className":26962},[508,545],[47,26964,616],{"className":26965},[615],[47,26967,26969],{"className":26968},[525],[47,26970,26972],{"className":26971,"style":7308},[529],[47,26973],{},[47,26975,1201],{"className":26976},[702],[47,26978],{"className":26979},[888,22015],[47,26981,26983,26986,27037,27040,27080,27083,27086,27089,27092,27095,27098],{"className":26982},[499],[47,26984],{"className":26985,"style":23653},[503],[47,26987,26989,26992],{"className":26988},[508],[47,26990,1163],{"className":26991},[508,581],[47,26993,26995],{"className":26994},[585],[47,26996,26998,27029],{"className":26997},[520,521],[47,26999,27001,27026],{"className":27000},[525],[47,27002,27004,27015],{"className":27003,"style":23672},[529],[47,27005,27006,27009],{"style":17673},[47,27007],{"className":27008,"style":929},[537],[47,27010,27012],{"className":27011},[542,543,544,545],[47,27013,14413],{"className":27014,"style":3580},[508,581,545],[47,27016,27017,27020],{"style":16788},[47,27018],{"className":27019,"style":929},[537],[47,27021,27023],{"className":27022},[542,543,544,545],[47,27024,472],{"className":27025},[508,545],[47,27027,616],{"className":27028},[615],[47,27030,27032],{"className":27031},[525],[47,27033,27035],{"className":27034,"style":17703},[529],[47,27036],{},[47,27038,1187],{"className":27039},[512],[47,27041,27043,27046],{"className":27042},[508],[47,27044,13164],{"className":27045},[508,581],[47,27047,27049],{"className":27048},[585],[47,27050,27052,27072],{"className":27051},[520,521],[47,27053,27055,27069],{"className":27054},[525],[47,27056,27058],{"className":27057,"style":13094},[529],[47,27059,27060,27063],{"style":1638},[47,27061],{"className":27062,"style":929},[537],[47,27064,27066],{"className":27065},[542,543,544,545],[47,27067,1166],{"className":27068},[508,581,545],[47,27070,616],{"className":27071},[615],[47,27073,27075],{"className":27074},[525],[47,27076,27078],{"className":27077,"style":1657},[529],[47,27079],{},[47,27081,1192],{"className":27082},[1299],[47,27084],{"className":27085,"style":1303},[888],[47,27087,285],{"className":27088},[508,581],[47,27090,1201],{"className":27091},[702],[47,27093],{"className":27094,"style":889},[888],[47,27096,82],{"className":27097},[893],[47,27099],{"className":27100,"style":889},[888],[47,27102,27104,27107],{"className":27103},[499],[47,27105],{"className":27106,"style":26267},[503],[47,27108,27110,27113],{"className":27109},[508],[47,27111,25839],{"className":27112,"style":25860},[508,581],[47,27114,27116],{"className":27115},[585],[47,27117,27119,27150],{"className":27118},[520,521],[47,27120,27122,27147],{"className":27121},[525],[47,27123,27125,27136],{"className":27124,"style":25158},[529],[47,27126,27127,27130],{"style":26288},[47,27128],{"className":27129,"style":929},[537],[47,27131,27133],{"className":27132},[542,543,544,545],[47,27134,1166],{"className":27135},[508,581,545],[47,27137,27138,27141],{"style":16788},[47,27139],{"className":27140,"style":929},[537],[47,27142,27144],{"className":27143},[542,543,544,545],[47,27145,1061],{"className":27146,"style":1091},[508,581,545],[47,27148,616],{"className":27149},[615],[47,27151,27153],{"className":27152},[525],[47,27154,27156],{"className":27155,"style":17703},[529],[47,27157],{},[47,27159],{"className":27160},[888,22015],[47,27162,27164,27167,27218,27221,27261,27264,27267,27270,27273,27276,27279],{"className":27163},[499],[47,27165],{"className":27166,"style":23653},[503],[47,27168,27170,27173],{"className":27169},[508],[47,27171,1163],{"className":27172},[508,581],[47,27174,27176],{"className":27175},[585],[47,27177,27179,27210],{"className":27178},[520,521],[47,27180,27182,27207],{"className":27181},[525],[47,27183,27185,27196],{"className":27184,"style":23672},[529],[47,27186,27187,27190],{"style":17673},[47,27188],{"className":27189,"style":929},[537],[47,27191,27193],{"className":27192},[542,543,544,545],[47,27194,14413],{"className":27195,"style":3580},[508,581,545],[47,27197,27198,27201],{"style":16788},[47,27199],{"className":27200,"style":929},[537],[47,27202,27204],{"className":27203},[542,543,544,545],[47,27205,3192],{"className":27206},[508,545],[47,27208,616],{"className":27209},[615],[47,27211,27213],{"className":27212},[525],[47,27214,27216],{"className":27215,"style":17703},[529],[47,27217],{},[47,27219,1187],{"className":27220},[512],[47,27222,27224,27227],{"className":27223},[508],[47,27225,13164],{"className":27226},[508,581],[47,27228,27230],{"className":27229},[585],[47,27231,27233,27253],{"className":27232},[520,521],[47,27234,27236,27250],{"className":27235},[525],[47,27237,27239],{"className":27238,"style":13094},[529],[47,27240,27241,27244],{"style":1638},[47,27242],{"className":27243,"style":929},[537],[47,27245,27247],{"className":27246},[542,543,544,545],[47,27248,1166],{"className":27249},[508,581,545],[47,27251,616],{"className":27252},[615],[47,27254,27256],{"className":27255},[525],[47,27257,27259],{"className":27258,"style":1657},[529],[47,27260],{},[47,27262,1192],{"className":27263},[1299],[47,27265],{"className":27266,"style":1303},[888],[47,27268,285],{"className":27269},[508,581],[47,27271,1201],{"className":27272},[702],[47,27274],{"className":27275,"style":889},[888],[47,27277,82],{"className":27278},[893],[47,27280],{"className":27281,"style":889},[888],[47,27283,27285,27288,27339,27342,27345],{"className":27284},[499],[47,27286],{"className":27287,"style":26267},[503],[47,27289,27291,27294],{"className":27290},[508],[47,27292,25839],{"className":27293,"style":25860},[508,581],[47,27295,27297],{"className":27296},[585],[47,27298,27300,27331],{"className":27299},[520,521],[47,27301,27303,27328],{"className":27302},[525],[47,27304,27306,27317],{"className":27305,"style":25158},[529],[47,27307,27308,27311],{"style":26288},[47,27309],{"className":27310,"style":929},[537],[47,27312,27314],{"className":27313},[542,543,544,545],[47,27315,1166],{"className":27316},[508,581,545],[47,27318,27319,27322],{"style":16788},[47,27320],{"className":27321,"style":929},[537],[47,27323,27325],{"className":27324},[542,543,544,545],[47,27326,1061],{"className":27327,"style":1091},[508,581,545],[47,27329,616],{"className":27330},[615],[47,27332,27334],{"className":27333},[525],[47,27335,27337],{"className":27336,"style":17703},[529],[47,27338],{},[47,27340],{"className":27341,"style":2259},[888],[47,27343,6147],{"className":27344},[942],[47,27346],{"className":27347,"style":2259},[888],[47,27349,27351,27354,27357],{"className":27350},[499],[47,27352],{"className":27353,"style":26613},[503],[47,27355,19357],{"className":27356,"style":19604},[508,581],[47,27358,27360,27363],{"className":27359},[508],[47,27361,25839],{"className":27362,"style":25860},[508,581],[47,27364,27366],{"className":27365},[585],[47,27367,27369,27409],{"className":27368},[520,521],[47,27370,27372,27406],{"className":27371},[525],[47,27373,27375,27395],{"className":27374,"style":25158},[529],[47,27376,27377,27380],{"style":26288},[47,27378],{"className":27379,"style":929},[537],[47,27381,27383],{"className":27382},[542,543,544,545],[47,27384,27386,27389,27392],{"className":27385},[508,545],[47,27387,1166],{"className":27388},[508,581,545],[47,27390,6147],{"className":27391},[942,545],[47,27393,472],{"className":27394},[508,545],[47,27396,27397,27400],{"style":16788},[47,27398],{"className":27399,"style":929},[537],[47,27401,27403],{"className":27402},[542,543,544,545],[47,27404,1061],{"className":27405,"style":1091},[508,581,545],[47,27407,616],{"className":27408},[615],[47,27410,27412],{"className":27411},[525],[47,27413,27415],{"className":27414,"style":26672},[529],[47,27416],{},[47,27418],{"className":27419},[888,22015],[47,27421,27423,27426,27477,27480,27520,27523,27526,27529,27532,27535,27538],{"className":27422},[499],[47,27424],{"className":27425,"style":23653},[503],[47,27427,27429,27432],{"className":27428},[508],[47,27430,1163],{"className":27431},[508,581],[47,27433,27435],{"className":27434},[585],[47,27436,27438,27469],{"className":27437},[520,521],[47,27439,27441,27466],{"className":27440},[525],[47,27442,27444,27455],{"className":27443,"style":23672},[529],[47,27445,27446,27449],{"style":17673},[47,27447],{"className":27448,"style":929},[537],[47,27450,27452],{"className":27451},[542,543,544,545],[47,27453,14413],{"className":27454,"style":3580},[508,581,545],[47,27456,27457,27460],{"style":16788},[47,27458],{"className":27459,"style":929},[537],[47,27461,27463],{"className":27462},[542,543,544,545],[47,27464,5041],{"className":27465},[508,545],[47,27467,616],{"className":27468},[615],[47,27470,27472],{"className":27471},[525],[47,27473,27475],{"className":27474,"style":17703},[529],[47,27476],{},[47,27478,1187],{"className":27479},[512],[47,27481,27483,27486],{"className":27482},[508],[47,27484,13164],{"className":27485},[508,581],[47,27487,27489],{"className":27488},[585],[47,27490,27492,27512],{"className":27491},[520,521],[47,27493,27495,27509],{"className":27494},[525],[47,27496,27498],{"className":27497,"style":13094},[529],[47,27499,27500,27503],{"style":1638},[47,27501],{"className":27502,"style":929},[537],[47,27504,27506],{"className":27505},[542,543,544,545],[47,27507,1166],{"className":27508},[508,581,545],[47,27510,616],{"className":27511},[615],[47,27513,27515],{"className":27514},[525],[47,27516,27518],{"className":27517,"style":1657},[529],[47,27519],{},[47,27521,1192],{"className":27522},[1299],[47,27524],{"className":27525,"style":1303},[888],[47,27527,285],{"className":27528},[508,581],[47,27530,1201],{"className":27531},[702],[47,27533],{"className":27534,"style":889},[888],[47,27536,82],{"className":27537},[893],[47,27539],{"className":27540,"style":889},[888],[47,27542,27544,27547,27598,27601,27604],{"className":27543},[499],[47,27545],{"className":27546,"style":26267},[503],[47,27548,27550,27553],{"className":27549},[508],[47,27551,25839],{"className":27552,"style":25860},[508,581],[47,27554,27556],{"className":27555},[585],[47,27557,27559,27590],{"className":27558},[520,521],[47,27560,27562,27587],{"className":27561},[525],[47,27563,27565,27576],{"className":27564,"style":25158},[529],[47,27566,27567,27570],{"style":26288},[47,27568],{"className":27569,"style":929},[537],[47,27571,27573],{"className":27572},[542,543,544,545],[47,27574,1166],{"className":27575},[508,581,545],[47,27577,27578,27581],{"style":16788},[47,27579],{"className":27580,"style":929},[537],[47,27582,27584],{"className":27583},[542,543,544,545],[47,27585,1061],{"className":27586,"style":1091},[508,581,545],[47,27588,616],{"className":27589},[615],[47,27591,27593],{"className":27592},[525],[47,27594,27596],{"className":27595,"style":17703},[529],[47,27597],{},[47,27599],{"className":27600,"style":2259},[888],[47,27602,6147],{"className":27603},[942],[47,27605],{"className":27606,"style":2259},[888],[47,27608,27610,27613,27616,27676,27679,27682],{"className":27609},[499],[47,27611],{"className":27612,"style":26613},[503],[47,27614,19357],{"className":27615,"style":19604},[508,581],[47,27617,27619,27622],{"className":27618},[508],[47,27620,25839],{"className":27621,"style":25860},[508,581],[47,27623,27625],{"className":27624},[585],[47,27626,27628,27668],{"className":27627},[520,521],[47,27629,27631,27665],{"className":27630},[525],[47,27632,27634,27654],{"className":27633,"style":25158},[529],[47,27635,27636,27639],{"style":26288},[47,27637],{"className":27638,"style":929},[537],[47,27640,27642],{"className":27641},[542,543,544,545],[47,27643,27645,27648,27651],{"className":27644},[508,545],[47,27646,1166],{"className":27647},[508,581,545],[47,27649,6147],{"className":27650},[942,545],[47,27652,472],{"className":27653},[508,545],[47,27655,27656,27659],{"style":16788},[47,27657],{"className":27658,"style":929},[537],[47,27660,27662],{"className":27661},[542,543,544,545],[47,27663,1061],{"className":27664,"style":1091},[508,581,545],[47,27666,616],{"className":27667},[615],[47,27669,27671],{"className":27670},[525],[47,27672,27674],{"className":27673,"style":26672},[529],[47,27675],{},[47,27677],{"className":27678,"style":2259},[888],[47,27680,6147],{"className":27681},[942],[47,27683],{"className":27684,"style":2259},[888],[47,27686,27688,27691,27720],{"className":27687},[499],[47,27689],{"className":27690,"style":26613},[503],[47,27692,27694,27697],{"className":27693},[508],[47,27695,19357],{"className":27696,"style":19604},[508,581],[47,27698,27700],{"className":27699},[585],[47,27701,27703],{"className":27702},[520],[47,27704,27706],{"className":27705},[525],[47,27707,27709],{"className":27708,"style":23672},[529],[47,27710,27711,27714],{"style":16788},[47,27712],{"className":27713,"style":929},[537],[47,27715,27717],{"className":27716},[542,543,544,545],[47,27718,3192],{"className":27719},[508,545],[47,27721,27723,27726],{"className":27722},[508],[47,27724,25839],{"className":27725,"style":25860},[508,581],[47,27727,27729],{"className":27728},[585],[47,27730,27732,27772],{"className":27731},[520,521],[47,27733,27735,27769],{"className":27734},[525],[47,27736,27738,27758],{"className":27737,"style":25158},[529],[47,27739,27740,27743],{"style":26288},[47,27741],{"className":27742,"style":929},[537],[47,27744,27746],{"className":27745},[542,543,544,545],[47,27747,27749,27752,27755],{"className":27748},[508,545],[47,27750,1166],{"className":27751},[508,581,545],[47,27753,6147],{"className":27754},[942,545],[47,27756,3192],{"className":27757},[508,545],[47,27759,27760,27763],{"style":16788},[47,27761],{"className":27762,"style":929},[537],[47,27764,27766],{"className":27765},[542,543,544,545],[47,27767,1061],{"className":27768,"style":1091},[508,581,545],[47,27770,616],{"className":27771},[615],[47,27773,27775],{"className":27774},[525],[47,27776,27778],{"className":27777,"style":26672},[529],[47,27779],{},[47,27781],{"className":27782},[888,22015],[47,27784,27786,27789],{"className":27785},[499],[47,27787],{"className":27788,"style":25118},[503],[47,27790,27792,27795],{"className":27791},[508],[47,27793,23500],{"className":27794},[508],[47,27796],{"className":27797,"style":25129},[508,25128],[207,27799,27801],{"id":27800},"generalized-advantage-estimation-gae","Generalized Advantage Estimation (GAE)",[14,27803,27804],{},"这时我们考虑，在运行时我们应该向后采样几步呢？GAE 的答案是，小孩子才做选择，我全都要！",[14,27806,27807,27808,27838],{},"GAE考虑全部的采样结果，并引入一个衰减因子 ",[47,27809,27811,27826],{"className":27810},[451],[47,27812,27814],{"className":27813},[455],[457,27815,27816],{"xmlns":459},[461,27817,27818,27823],{},[464,27819,27820],{},[480,27821,27822],{},"λ",[487,27824,27825],{"encoding":489},"\\lambda",[47,27827,27829],{"className":27828,"ariaHidden":495},[494],[47,27830,27832,27835],{"className":27831},[499],[47,27833],{"className":27834,"style":796},[503],[47,27836,27822],{"className":27837},[508,581]," 。",[47,27840,27842],{"className":27841},[14712],[47,27843,27845,28436],{"className":27844},[451],[47,27846,27848],{"className":27847},[455],[457,27849,27850],{"xmlns":459,"display":9848},[461,27851,27852,28433],{},[14724,27853,27854,28032,28174,28284,28374],{"rowspacing":14726,"columnalign":14727,"columnspacing":14728},[14730,27855,27856,27891],{},[14733,27857,27858],{},[14736,27859,27860],{"scriptlevel":2141,"displaystyle":495},[464,27861,27862,27877,27879,27885,27887,27889],{},[2209,27863,27864,27866,27868],{},[480,27865,1163],{},[480,27867,14413],{},[464,27869,27870,27873,27875],{},[480,27871,27872],{},"G",[480,27874,1163],{},[480,27876,5977],{},[850,27878,1187],{"stretchy":1186},[477,27880,27881,27883],{},[480,27882,13164],{},[480,27884,1166],{},[850,27886,1192],{"separator":495},[480,27888,285],{},[850,27890,1201],{"stretchy":1186},[14733,27892,27893],{},[14736,27894,27895],{"scriptlevel":2141,"displaystyle":495},[464,27896,27897,27899,27901,27903,27905,27907,27909,27911,27913,27921,27923,27925,27927,27935,27937,27943,27951,27953,27955,27958,27960,27963,27965,27967,27969,27972,27974,27976,27990,27992,27994,28002,28004,28007,28015,28017,28020,28028,28030],{},[464,27898],{},[850,27900,82],{},[850,27902,1187],{"stretchy":1186},[470,27904,472],{},[850,27906,18727],{},[480,27908,27822],{},[850,27910,1201],{"stretchy":1186},[850,27912,1187],{"stretchy":1186},[2209,27914,27915,27917,27919],{},[480,27916,1163],{},[480,27918,14413],{},[470,27920,472],{},[850,27922,6147],{},[480,27924,27822],{},[850,27926,23411],{},[2209,27928,27929,27931,27933],{},[480,27930,1163],{},[480,27932,14413],{},[470,27934,3192],{},[850,27936,6147],{},[854,27938,27939,27941],{},[480,27940,27822],{},[470,27942,3192],{},[2209,27944,27945,27947,27949],{},[480,27946,1163],{},[480,27948,14413],{},[470,27950,5041],{},[850,27952,6147],{},[850,27954,23606],{},[3647,27956,27957],{}," ",[850,27959,1201],{"stretchy":1186},[888,27961],{"width":27962},"1em",[888,27964],{"width":27962},[480,27966,27822],{},[850,27968,82],{},[470,27970,27971],{},"0.9",[850,27973,7554],{},[888,27975],{"width":27962},[2209,27977,27978,27980,27982],{},[480,27979,1163],{},[480,27981,14413],{},[464,27983,27984,27986,27988],{},[480,27985,27872],{},[480,27987,1163],{},[480,27989,5977],{},[850,27991,82],{},[470,27993,12830],{},[2209,27995,27996,27998,28000],{},[480,27997,1163],{},[480,27999,14413],{},[470,28001,472],{},[850,28003,6147],{},[470,28005,28006],{},"0.09",[2209,28008,28009,28011,28013],{},[480,28010,1163],{},[480,28012,14413],{},[470,28014,3192],{},[850,28016,6147],{},[470,28018,28019],{},"0.081",[2209,28021,28022,28024,28026],{},[480,28023,1163],{},[480,28025,14413],{},[470,28027,5041],{},[850,28029,6147],{},[850,28031,23606],{},[14730,28033,28034,28040],{},[14733,28035,28036],{},[14736,28037,28038],{"scriptlevel":2141,"displaystyle":495},[464,28039],{},[14733,28041,28042],{},[14736,28043,28044],{"scriptlevel":2141,"displaystyle":495},[464,28045,28046,28048,28050,28052,28054,28056,28058,28060,28062,28070,28072,28074,28076,28078,28086,28088,28090,28104,28106,28108,28114,28116,28124,28126,28128,28142,28144,28150,28164,28166,28168,28170,28172],{},[464,28047],{},[850,28049,82],{},[850,28051,1187],{"stretchy":1186},[470,28053,472],{},[850,28055,18727],{},[480,28057,27822],{},[850,28059,1201],{"stretchy":1186},[850,28061,1187],{"stretchy":1186},[2209,28063,28064,28066,28068],{},[480,28065,25839],{},[480,28067,1166],{},[480,28069,1061],{},[850,28071,6147],{},[480,28073,27822],{},[850,28075,23411],{},[850,28077,1187],{"stretchy":1186},[2209,28079,28080,28082,28084],{},[480,28081,25839],{},[480,28083,1166],{},[480,28085,1061],{},[850,28087,6147],{},[480,28089,19357],{},[2209,28091,28092,28094,28102],{},[480,28093,25839],{},[464,28095,28096,28098,28100],{},[480,28097,1166],{},[850,28099,6147],{},[470,28101,472],{},[480,28103,1061],{},[850,28105,1201],{"stretchy":1186},[850,28107,6147],{},[854,28109,28110,28112],{},[480,28111,27822],{},[470,28113,3192],{},[850,28115,1187],{"stretchy":1186},[2209,28117,28118,28120,28122],{},[480,28119,25839],{},[480,28121,1166],{},[480,28123,1061],{},[850,28125,6147],{},[480,28127,19357],{},[2209,28129,28130,28132,28140],{},[480,28131,25839],{},[464,28133,28134,28136,28138],{},[480,28135,1166],{},[850,28137,6147],{},[470,28139,472],{},[480,28141,1061],{},[850,28143,6147],{},[854,28145,28146,28148],{},[480,28147,19357],{},[470,28149,3192],{},[2209,28151,28152,28154,28162],{},[480,28153,25839],{},[464,28155,28156,28158,28160],{},[480,28157,1166],{},[850,28159,6147],{},[470,28161,3192],{},[480,28163,1061],{},[850,28165,1201],{"stretchy":1186},[850,28167,6147],{},[850,28169,23606],{},[3647,28171,27957],{},[850,28173,1201],{"stretchy":1186},[14730,28175,28176,28182],{},[14733,28177,28178],{},[14736,28179,28180],{"scriptlevel":2141,"displaystyle":495},[464,28181],{},[14733,28183,28184],{},[14736,28185,28186],{"scriptlevel":2141,"displaystyle":495},[464,28187,28188,28190,28192,28194,28196,28198,28200,28202,28204,28212,28214,28216,28218,28220,28222,28228,28230,28232,28234,28236,28238,28240,28254,28256,28258,28260,28262,28268,28270,28272,28274,28276,28278,28280,28282],{},[464,28189],{},[850,28191,82],{},[850,28193,1187],{"stretchy":1186},[470,28195,472],{},[850,28197,18727],{},[480,28199,27822],{},[850,28201,1201],{"stretchy":1186},[850,28203,1187],{"stretchy":1186},[2209,28205,28206,28208,28210],{},[480,28207,25839],{},[480,28209,1166],{},[480,28211,1061],{},[850,28213,1187],{"stretchy":1186},[470,28215,472],{},[850,28217,6147],{},[480,28219,27822],{},[850,28221,6147],{},[854,28223,28224,28226],{},[480,28225,27822],{},[470,28227,3192],{},[850,28229,6147],{},[850,28231,23606],{},[3647,28233,27957],{},[850,28235,1201],{"stretchy":1186},[850,28237,6147],{},[480,28239,19357],{},[2209,28241,28242,28244,28252],{},[480,28243,25839],{},[464,28245,28246,28248,28250],{},[480,28247,1166],{},[850,28249,6147],{},[470,28251,472],{},[480,28253,1061],{},[850,28255,23411],{},[850,28257,1187],{"stretchy":1186},[480,28259,27822],{},[850,28261,6147],{},[854,28263,28264,28266],{},[480,28265,27822],{},[470,28267,3192],{},[850,28269,6147],{},[850,28271,23606],{},[3647,28273,27957],{},[850,28275,1201],{"stretchy":1186},[850,28277,6147],{},[850,28279,23606],{},[3647,28281,27957],{},[850,28283,1201],{"stretchy":1186},[14730,28285,28286,28292],{},[14733,28287,28288],{},[14736,28289,28290],{"scriptlevel":2141,"displaystyle":495},[464,28291],{},[14733,28293,28294],{},[14736,28295,28296],{"scriptlevel":2141,"displaystyle":495},[464,28297,28298,28300,28302,28304,28306,28308,28310,28312],{},[464,28299],{},[850,28301,82],{},[850,28303,1187],{"stretchy":1186},[470,28305,472],{},[850,28307,18727],{},[480,28309,27822],{},[850,28311,1201],{"stretchy":1186},[464,28313,28314,28316,28324,28336,28338,28340,28354,28366,28368,28370,28372],{},[850,28315,1187],{"fence":495},[2209,28317,28318,28320,28322],{},[480,28319,25839],{},[480,28321,1166],{},[480,28323,1061],{},[467,28325,28326,28328],{},[470,28327,472],{},[464,28329,28330,28332,28334],{},[470,28331,472],{},[850,28333,18727],{},[480,28335,27822],{},[850,28337,6147],{},[480,28339,19357],{},[2209,28341,28342,28344,28352],{},[480,28343,25839],{},[464,28345,28346,28348,28350],{},[480,28347,1166],{},[850,28349,6147],{},[470,28351,472],{},[480,28353,1061],{},[467,28355,28356,28358],{},[480,28357,27822],{},[464,28359,28360,28362,28364],{},[470,28361,472],{},[850,28363,18727],{},[480,28365,27822],{},[850,28367,6147],{},[850,28369,23606],{},[3647,28371,27957],{},[850,28373,1201],{"fence":495},[14730,28375,28376,28382],{},[14733,28377,28378],{},[14736,28379,28380],{"scriptlevel":2141,"displaystyle":495},[464,28381],{},[14733,28383,28384],{},[14736,28385,28386],{"scriptlevel":2141,"displaystyle":495},[464,28387,28388,28390,28392,28407,28409,28411,28413,28419],{},[464,28389],{},[850,28391,82],{},[15045,28393,28394,28396,28404],{},[850,28395,2213],{},[464,28397,28398,28400,28402],{},[480,28399,7727],{},[850,28401,82],{},[470,28403,2141],{},[480,28405,28406],{"mathvariant":1206},"∞",[850,28408,1187],{"stretchy":1186},[480,28410,19357],{},[480,28412,27822],{},[854,28414,28415,28417],{},[850,28416,1201],{"stretchy":1186},[480,28418,7727],{},[2209,28420,28421,28423,28431],{},[480,28422,25839],{},[464,28424,28425,28427,28429],{},[480,28426,1166],{},[850,28428,6147],{},[480,28430,7727],{},[480,28432,1061],{},[487,28434,28435],{"encoding":489},"\\begin{aligned}\nA_\\theta^{GAE}(s_t, a) &= (1 - \\lambda)(A_\\theta^1 + \\lambda * A_\\theta^2 + \\lambda^2 A_\\theta^3 + \\cdots) \\quad\\quad \\lambda = 0.9: \\quad A_\\theta^{GAE} = 0.1A_\\theta^1 + 0.09A_\\theta^2 + 0.081A_\\theta^3 + \\cdots \\\\\n&= (1 - \\lambda)(\\delta_t^V + \\lambda * (\\delta_t^V + \\gamma \\delta_{t+1}^V) + \\lambda^2 (\\delta_t^V + \\gamma \\delta_{t+1}^V + \\gamma^2 \\delta_{t+2}^V) + \\cdots) \\\\\n&= (1 - \\lambda)(\\delta_t^V (1 + \\lambda + \\lambda^2 + \\cdots) + \\gamma \\delta_{t+1}^V * (\\lambda + \\lambda^2 + \\cdots) + \\cdots) \\\\\n&= (1 - \\lambda)\\left(\\delta_t^V \\frac{1}{1 - \\lambda} + \\gamma \\delta_{t+1}^V \\frac{\\lambda}{1 - \\lambda} + \\cdots\\right) \\\\\n&= \\sum\\limits_{b=0}^\\infty (\\gamma\\lambda)^b \\delta_{t+b}^V\n\\end{aligned}",[47,28437,28439],{"className":28438,"ariaHidden":495},[494],[47,28440,28442,28446],{"className":28441},[499],[47,28443],{"className":28444,"style":28445},[503],"height:10.6075em;vertical-align:-5.0538em;",[47,28447,28449],{"className":28448},[508],[47,28450,28452,28638],{"className":28451},[14724],[47,28453,28455],{"className":28454},[15515],[47,28456,28458,28629],{"className":28457},[520,521],[47,28459,28461,28626],{"className":28460},[525],[47,28462,28465,28590,28599,28608,28617],{"className":28463,"style":28464},[529],"height:5.5538em;",[47,28466,28468,28472],{"style":28467},"top:-8.3138em;",[47,28469],{"className":28470,"style":28471},[537],"height:3.6514em;",[47,28473,28475,28535,28538,28578,28581,28584,28587],{"className":28474},[508],[47,28476,28478,28481],{"className":28477},[508],[47,28479,1163],{"className":28480},[508,581],[47,28482,28484],{"className":28483},[585],[47,28485,28487,28527],{"className":28486},[520,521],[47,28488,28490,28524],{"className":28489},[525],[47,28491,28493,28504],{"className":28492,"style":25158},[529],[47,28494,28495,28498],{"style":17673},[47,28496],{"className":28497,"style":929},[537],[47,28499,28501],{"className":28500},[542,543,544,545],[47,28502,14413],{"className":28503,"style":3580},[508,581,545],[47,28505,28506,28509],{"style":16788},[47,28507],{"className":28508,"style":929},[537],[47,28510,28512],{"className":28511},[542,543,544,545],[47,28513,28515,28518,28521],{"className":28514},[508,545],[47,28516,27872],{"className":28517},[508,581,545],[47,28519,1163],{"className":28520},[508,581,545],[47,28522,5977],{"className":28523,"style":6420},[508,581,545],[47,28525,616],{"className":28526},[615],[47,28528,28530],{"className":28529},[525],[47,28531,28533],{"className":28532,"style":17703},[529],[47,28534],{},[47,28536,1187],{"className":28537},[512],[47,28539,28541,28544],{"className":28540},[508],[47,28542,13164],{"className":28543},[508,581],[47,28545,28547],{"className":28546},[585],[47,28548,28550,28570],{"className":28549},[520,521],[47,28551,28553,28567],{"className":28552},[525],[47,28554,28556],{"className":28555,"style":13094},[529],[47,28557,28558,28561],{"style":1638},[47,28559],{"className":28560,"style":929},[537],[47,28562,28564],{"className":28563},[542,543,544,545],[47,28565,1166],{"className":28566},[508,581,545],[47,28568,616],{"className":28569},[615],[47,28571,28573],{"className":28572},[525],[47,28574,28576],{"className":28575,"style":1657},[529],[47,28577],{},[47,28579,1192],{"className":28580},[1299],[47,28582],{"className":28583,"style":1303},[888],[47,28585,285],{"className":28586},[508,581],[47,28588,1201],{"className":28589},[702],[47,28591,28593,28596],{"style":28592},"top:-6.7625em;",[47,28594],{"className":28595,"style":28471},[537],[47,28597],{"className":28598},[508],[47,28600,28602,28605],{"style":28601},"top:-5.2112em;",[47,28603],{"className":28604,"style":28471},[537],[47,28606],{"className":28607},[508],[47,28609,28611,28614],{"style":28610},"top:-3.1012em;",[47,28612],{"className":28613,"style":28471},[537],[47,28615],{"className":28616},[508],[47,28618,28620,28623],{"style":28619},"top:-0.1997em;",[47,28621],{"className":28622,"style":28471},[537],[47,28624],{"className":28625},[508],[47,28627,616],{"className":28628},[615],[47,28630,28632],{"className":28631},[525],[47,28633,28636],{"className":28634,"style":28635},[529],"height:5.0538em;",[47,28637],{},[47,28639,28641],{"className":28640},[15739],[47,28642,28644,30613],{"className":28643},[520,521],[47,28645,28647,30610],{"className":28646},[525],[47,28648,28650,29219,29747,30077,30424],{"className":28649,"style":28464},[529],[47,28651,28652,28655],{"style":28467},[47,28653],{"className":28654,"style":28471},[537],[47,28656,28658,28661,28664,28667,28670,28673,28676,28679,28682,28685,28688,28691,28694,28745,28748,28751,28754,28757,28760,28763,28766,28817,28820,28823,28826,28855,28906,28909,28912,28915,28918,28921,28924,28928,28931,28934,28937,28940,28943,28946,28949,28952,28955,28958,29018,29021,29024,29027,29030,29081,29084,29087,29090,29093,29144,29147,29150,29153,29156,29207,29210,29213,29216],{"className":28657},[508],[47,28659],{"className":28660},[508],[47,28662],{"className":28663,"style":889},[888],[47,28665,82],{"className":28666},[893],[47,28668],{"className":28669,"style":889},[888],[47,28671,1187],{"className":28672},[512],[47,28674,472],{"className":28675},[508],[47,28677],{"className":28678,"style":2259},[888],[47,28680,18727],{"className":28681},[942],[47,28683],{"className":28684,"style":2259},[888],[47,28686,27822],{"className":28687},[508,581],[47,28689,1201],{"className":28690},[702],[47,28692,1187],{"className":28693},[512],[47,28695,28697,28700],{"className":28696},[508],[47,28698,1163],{"className":28699},[508,581],[47,28701,28703],{"className":28702},[585],[47,28704,28706,28737],{"className":28705},[520,521],[47,28707,28709,28734],{"className":28708},[525],[47,28710,28712,28723],{"className":28711,"style":23672},[529],[47,28713,28714,28717],{"style":17673},[47,28715],{"className":28716,"style":929},[537],[47,28718,28720],{"className":28719},[542,543,544,545],[47,28721,14413],{"className":28722,"style":3580},[508,581,545],[47,28724,28725,28728],{"style":16788},[47,28726],{"className":28727,"style":929},[537],[47,28729,28731],{"className":28730},[542,543,544,545],[47,28732,472],{"className":28733},[508,545],[47,28735,616],{"className":28736},[615],[47,28738,28740],{"className":28739},[525],[47,28741,28743],{"className":28742,"style":17703},[529],[47,28744],{},[47,28746],{"className":28747,"style":2259},[888],[47,28749,6147],{"className":28750},[942],[47,28752],{"className":28753,"style":2259},[888],[47,28755,27822],{"className":28756},[508,581],[47,28758],{"className":28759,"style":2259},[888],[47,28761,23411],{"className":28762},[942],[47,28764],{"className":28765,"style":2259},[888],[47,28767,28769,28772],{"className":28768},[508],[47,28770,1163],{"className":28771},[508,581],[47,28773,28775],{"className":28774},[585],[47,28776,28778,28809],{"className":28777},[520,521],[47,28779,28781,28806],{"className":28780},[525],[47,28782,28784,28795],{"className":28783,"style":23672},[529],[47,28785,28786,28789],{"style":17673},[47,28787],{"className":28788,"style":929},[537],[47,28790,28792],{"className":28791},[542,543,544,545],[47,28793,14413],{"className":28794,"style":3580},[508,581,545],[47,28796,28797,28800],{"style":16788},[47,28798],{"className":28799,"style":929},[537],[47,28801,28803],{"className":28802},[542,543,544,545],[47,28804,3192],{"className":28805},[508,545],[47,28807,616],{"className":28808},[615],[47,28810,28812],{"className":28811},[525],[47,28813,28815],{"className":28814,"style":17703},[529],[47,28816],{},[47,28818],{"className":28819,"style":2259},[888],[47,28821,6147],{"className":28822},[942],[47,28824],{"className":28825,"style":2259},[888],[47,28827,28829,28832],{"className":28828},[508],[47,28830,27822],{"className":28831},[508,581],[47,28833,28835],{"className":28834},[585],[47,28836,28838],{"className":28837},[520],[47,28839,28841],{"className":28840},[525],[47,28842,28844],{"className":28843,"style":23672},[529],[47,28845,28846,28849],{"style":16788},[47,28847],{"className":28848,"style":929},[537],[47,28850,28852],{"className":28851},[542,543,544,545],[47,28853,3192],{"className":28854},[508,545],[47,28856,28858,28861],{"className":28857},[508],[47,28859,1163],{"className":28860},[508,581],[47,28862,28864],{"className":28863},[585],[47,28865,28867,28898],{"className":28866},[520,521],[47,28868,28870,28895],{"className":28869},[525],[47,28871,28873,28884],{"className":28872,"style":23672},[529],[47,28874,28875,28878],{"style":17673},[47,28876],{"className":28877,"style":929},[537],[47,28879,28881],{"className":28880},[542,543,544,545],[47,28882,14413],{"className":28883,"style":3580},[508,581,545],[47,28885,28886,28889],{"style":16788},[47,28887],{"className":28888,"style":929},[537],[47,28890,28892],{"className":28891},[542,543,544,545],[47,28893,5041],{"className":28894},[508,545],[47,28896,616],{"className":28897},[615],[47,28899,28901],{"className":28900},[525],[47,28902,28904],{"className":28903,"style":17703},[529],[47,28905],{},[47,28907],{"className":28908,"style":2259},[888],[47,28910,6147],{"className":28911},[942],[47,28913],{"className":28914,"style":2259},[888],[47,28916,23606],{"className":28917},[1349],[47,28919],{"className":28920,"style":1303},[888],[47,28922,1201],{"className":28923},[702],[47,28925],{"className":28926,"style":28927},[888],"margin-right:1em;",[47,28929],{"className":28930,"style":28927},[888],[47,28932,27822],{"className":28933},[508,581],[47,28935],{"className":28936,"style":889},[888],[47,28938,82],{"className":28939},[893],[47,28941],{"className":28942,"style":889},[888],[47,28944,27971],{"className":28945},[508],[47,28947],{"className":28948,"style":889},[888],[47,28950,7554],{"className":28951},[893],[47,28953],{"className":28954,"style":28927},[888],[47,28956],{"className":28957,"style":889},[888],[47,28959,28961,28964],{"className":28960},[508],[47,28962,1163],{"className":28963},[508,581],[47,28965,28967],{"className":28966},[585],[47,28968,28970,29010],{"className":28969},[520,521],[47,28971,28973,29007],{"className":28972},[525],[47,28974,28976,28987],{"className":28975,"style":25158},[529],[47,28977,28978,28981],{"style":17673},[47,28979],{"className":28980,"style":929},[537],[47,28982,28984],{"className":28983},[542,543,544,545],[47,28985,14413],{"className":28986,"style":3580},[508,581,545],[47,28988,28989,28992],{"style":16788},[47,28990],{"className":28991,"style":929},[537],[47,28993,28995],{"className":28994},[542,543,544,545],[47,28996,28998,29001,29004],{"className":28997},[508,545],[47,28999,27872],{"className":29000},[508,581,545],[47,29002,1163],{"className":29003},[508,581,545],[47,29005,5977],{"className":29006,"style":6420},[508,581,545],[47,29008,616],{"className":29009},[615],[47,29011,29013],{"className":29012},[525],[47,29014,29016],{"className":29015,"style":17703},[529],[47,29017],{},[47,29019],{"className":29020,"style":889},[888],[47,29022,82],{"className":29023},[893],[47,29025],{"className":29026,"style":889},[888],[47,29028,12830],{"className":29029},[508],[47,29031,29033,29036],{"className":29032},[508],[47,29034,1163],{"className":29035},[508,581],[47,29037,29039],{"className":29038},[585],[47,29040,29042,29073],{"className":29041},[520,521],[47,29043,29045,29070],{"className":29044},[525],[47,29046,29048,29059],{"className":29047,"style":23672},[529],[47,29049,29050,29053],{"style":17673},[47,29051],{"className":29052,"style":929},[537],[47,29054,29056],{"className":29055},[542,543,544,545],[47,29057,14413],{"className":29058,"style":3580},[508,581,545],[47,29060,29061,29064],{"style":16788},[47,29062],{"className":29063,"style":929},[537],[47,29065,29067],{"className":29066},[542,543,544,545],[47,29068,472],{"className":29069},[508,545],[47,29071,616],{"className":29072},[615],[47,29074,29076],{"className":29075},[525],[47,29077,29079],{"className":29078,"style":17703},[529],[47,29080],{},[47,29082],{"className":29083,"style":2259},[888],[47,29085,6147],{"className":29086},[942],[47,29088],{"className":29089,"style":2259},[888],[47,29091,28006],{"className":29092},[508],[47,29094,29096,29099],{"className":29095},[508],[47,29097,1163],{"className":29098},[508,581],[47,29100,29102],{"className":29101},[585],[47,29103,29105,29136],{"className":29104},[520,521],[47,29106,29108,29133],{"className":29107},[525],[47,29109,29111,29122],{"className":29110,"style":23672},[529],[47,29112,29113,29116],{"style":17673},[47,29114],{"className":29115,"style":929},[537],[47,29117,29119],{"className":29118},[542,543,544,545],[47,29120,14413],{"className":29121,"style":3580},[508,581,545],[47,29123,29124,29127],{"style":16788},[47,29125],{"className":29126,"style":929},[537],[47,29128,29130],{"className":29129},[542,543,544,545],[47,29131,3192],{"className":29132},[508,545],[47,29134,616],{"className":29135},[615],[47,29137,29139],{"className":29138},[525],[47,29140,29142],{"className":29141,"style":17703},[529],[47,29143],{},[47,29145],{"className":29146,"style":2259},[888],[47,29148,6147],{"className":29149},[942],[47,29151],{"className":29152,"style":2259},[888],[47,29154,28019],{"className":29155},[508],[47,29157,29159,29162],{"className":29158},[508],[47,29160,1163],{"className":29161},[508,581],[47,29163,29165],{"className":29164},[585],[47,29166,29168,29199],{"className":29167},[520,521],[47,29169,29171,29196],{"className":29170},[525],[47,29172,29174,29185],{"className":29173,"style":23672},[529],[47,29175,29176,29179],{"style":17673},[47,29177],{"className":29178,"style":929},[537],[47,29180,29182],{"className":29181},[542,543,544,545],[47,29183,14413],{"className":29184,"style":3580},[508,581,545],[47,29186,29187,29190],{"style":16788},[47,29188],{"className":29189,"style":929},[537],[47,29191,29193],{"className":29192},[542,543,544,545],[47,29194,5041],{"className":29195},[508,545],[47,29197,616],{"className":29198},[615],[47,29200,29202],{"className":29201},[525],[47,29203,29205],{"className":29204,"style":17703},[529],[47,29206],{},[47,29208],{"className":29209,"style":2259},[888],[47,29211,6147],{"className":29212},[942],[47,29214],{"className":29215,"style":2259},[888],[47,29217,23606],{"className":29218},[1349],[47,29220,29221,29224],{"style":28592},[47,29222],{"className":29223,"style":28471},[537],[47,29225,29227,29230,29233,29236,29239,29242,29245,29248,29251,29254,29257,29260,29263,29314,29317,29320,29323,29326,29329,29332,29335,29338,29389,29392,29395,29398,29401,29461,29464,29467,29470,29473,29502,29505,29556,29559,29562,29565,29568,29628,29631,29634,29637,29666,29726,29729,29732,29735,29738,29741,29744],{"className":29226},[508],[47,29228],{"className":29229},[508],[47,29231],{"className":29232,"style":889},[888],[47,29234,82],{"className":29235},[893],[47,29237],{"className":29238,"style":889},[888],[47,29240,1187],{"className":29241},[512],[47,29243,472],{"className":29244},[508],[47,29246],{"className":29247,"style":2259},[888],[47,29249,18727],{"className":29250},[942],[47,29252],{"className":29253,"style":2259},[888],[47,29255,27822],{"className":29256},[508,581],[47,29258,1201],{"className":29259},[702],[47,29261,1187],{"className":29262},[512],[47,29264,29266,29269],{"className":29265},[508],[47,29267,25839],{"className":29268,"style":25860},[508,581],[47,29270,29272],{"className":29271},[585],[47,29273,29275,29306],{"className":29274},[520,521],[47,29276,29278,29303],{"className":29277},[525],[47,29279,29281,29292],{"className":29280,"style":25158},[529],[47,29282,29283,29286],{"style":26288},[47,29284],{"className":29285,"style":929},[537],[47,29287,29289],{"className":29288},[542,543,544,545],[47,29290,1166],{"className":29291},[508,581,545],[47,29293,29294,29297],{"style":16788},[47,29295],{"className":29296,"style":929},[537],[47,29298,29300],{"className":29299},[542,543,544,545],[47,29301,1061],{"className":29302,"style":1091},[508,581,545],[47,29304,616],{"className":29305},[615],[47,29307,29309],{"className":29308},[525],[47,29310,29312],{"className":29311,"style":17703},[529],[47,29313],{},[47,29315],{"className":29316,"style":2259},[888],[47,29318,6147],{"className":29319},[942],[47,29321],{"className":29322,"style":2259},[888],[47,29324,27822],{"className":29325},[508,581],[47,29327],{"className":29328,"style":2259},[888],[47,29330,23411],{"className":29331},[942],[47,29333],{"className":29334,"style":2259},[888],[47,29336,1187],{"className":29337},[512],[47,29339,29341,29344],{"className":29340},[508],[47,29342,25839],{"className":29343,"style":25860},[508,581],[47,29345,29347],{"className":29346},[585],[47,29348,29350,29381],{"className":29349},[520,521],[47,29351,29353,29378],{"className":29352},[525],[47,29354,29356,29367],{"className":29355,"style":25158},[529],[47,29357,29358,29361],{"style":26288},[47,29359],{"className":29360,"style":929},[537],[47,29362,29364],{"className":29363},[542,543,544,545],[47,29365,1166],{"className":29366},[508,581,545],[47,29368,29369,29372],{"style":16788},[47,29370],{"className":29371,"style":929},[537],[47,29373,29375],{"className":29374},[542,543,544,545],[47,29376,1061],{"className":29377,"style":1091},[508,581,545],[47,29379,616],{"className":29380},[615],[47,29382,29384],{"className":29383},[525],[47,29385,29387],{"className":29386,"style":17703},[529],[47,29388],{},[47,29390],{"className":29391,"style":2259},[888],[47,29393,6147],{"className":29394},[942],[47,29396],{"className":29397,"style":2259},[888],[47,29399,19357],{"className":29400,"style":19604},[508,581],[47,29402,29404,29407],{"className":29403},[508],[47,29405,25839],{"className":29406,"style":25860},[508,581],[47,29408,29410],{"className":29409},[585],[47,29411,29413,29453],{"className":29412},[520,521],[47,29414,29416,29450],{"className":29415},[525],[47,29417,29419,29439],{"className":29418,"style":25158},[529],[47,29420,29421,29424],{"style":26288},[47,29422],{"className":29423,"style":929},[537],[47,29425,29427],{"className":29426},[542,543,544,545],[47,29428,29430,29433,29436],{"className":29429},[508,545],[47,29431,1166],{"className":29432},[508,581,545],[47,29434,6147],{"className":29435},[942,545],[47,29437,472],{"className":29438},[508,545],[47,29440,29441,29444],{"style":16788},[47,29442],{"className":29443,"style":929},[537],[47,29445,29447],{"className":29446},[542,543,544,545],[47,29448,1061],{"className":29449,"style":1091},[508,581,545],[47,29451,616],{"className":29452},[615],[47,29454,29456],{"className":29455},[525],[47,29457,29459],{"className":29458,"style":26672},[529],[47,29460],{},[47,29462,1201],{"className":29463},[702],[47,29465],{"className":29466,"style":2259},[888],[47,29468,6147],{"className":29469},[942],[47,29471],{"className":29472,"style":2259},[888],[47,29474,29476,29479],{"className":29475},[508],[47,29477,27822],{"className":29478},[508,581],[47,29480,29482],{"className":29481},[585],[47,29483,29485],{"className":29484},[520],[47,29486,29488],{"className":29487},[525],[47,29489,29491],{"className":29490,"style":23672},[529],[47,29492,29493,29496],{"style":16788},[47,29494],{"className":29495,"style":929},[537],[47,29497,29499],{"className":29498},[542,543,544,545],[47,29500,3192],{"className":29501},[508,545],[47,29503,1187],{"className":29504},[512],[47,29506,29508,29511],{"className":29507},[508],[47,29509,25839],{"className":29510,"style":25860},[508,581],[47,29512,29514],{"className":29513},[585],[47,29515,29517,29548],{"className":29516},[520,521],[47,29518,29520,29545],{"className":29519},[525],[47,29521,29523,29534],{"className":29522,"style":25158},[529],[47,29524,29525,29528],{"style":26288},[47,29526],{"className":29527,"style":929},[537],[47,29529,29531],{"className":29530},[542,543,544,545],[47,29532,1166],{"className":29533},[508,581,545],[47,29535,29536,29539],{"style":16788},[47,29537],{"className":29538,"style":929},[537],[47,29540,29542],{"className":29541},[542,543,544,545],[47,29543,1061],{"className":29544,"style":1091},[508,581,545],[47,29546,616],{"className":29547},[615],[47,29549,29551],{"className":29550},[525],[47,29552,29554],{"className":29553,"style":17703},[529],[47,29555],{},[47,29557],{"className":29558,"style":2259},[888],[47,29560,6147],{"className":29561},[942],[47,29563],{"className":29564,"style":2259},[888],[47,29566,19357],{"className":29567,"style":19604},[508,581],[47,29569,29571,29574],{"className":29570},[508],[47,29572,25839],{"className":29573,"style":25860},[508,581],[47,29575,29577],{"className":29576},[585],[47,29578,29580,29620],{"className":29579},[520,521],[47,29581,29583,29617],{"className":29582},[525],[47,29584,29586,29606],{"className":29585,"style":25158},[529],[47,29587,29588,29591],{"style":26288},[47,29589],{"className":29590,"style":929},[537],[47,29592,29594],{"className":29593},[542,543,544,545],[47,29595,29597,29600,29603],{"className":29596},[508,545],[47,29598,1166],{"className":29599},[508,581,545],[47,29601,6147],{"className":29602},[942,545],[47,29604,472],{"className":29605},[508,545],[47,29607,29608,29611],{"style":16788},[47,29609],{"className":29610,"style":929},[537],[47,29612,29614],{"className":29613},[542,543,544,545],[47,29615,1061],{"className":29616,"style":1091},[508,581,545],[47,29618,616],{"className":29619},[615],[47,29621,29623],{"className":29622},[525],[47,29624,29626],{"className":29625,"style":26672},[529],[47,29627],{},[47,29629],{"className":29630,"style":2259},[888],[47,29632,6147],{"className":29633},[942],[47,29635],{"className":29636,"style":2259},[888],[47,29638,29640,29643],{"className":29639},[508],[47,29641,19357],{"className":29642,"style":19604},[508,581],[47,29644,29646],{"className":29645},[585],[47,29647,29649],{"className":29648},[520],[47,29650,29652],{"className":29651},[525],[47,29653,29655],{"className":29654,"style":23672},[529],[47,29656,29657,29660],{"style":16788},[47,29658],{"className":29659,"style":929},[537],[47,29661,29663],{"className":29662},[542,543,544,545],[47,29664,3192],{"className":29665},[508,545],[47,29667,29669,29672],{"className":29668},[508],[47,29670,25839],{"className":29671,"style":25860},[508,581],[47,29673,29675],{"className":29674},[585],[47,29676,29678,29718],{"className":29677},[520,521],[47,29679,29681,29715],{"className":29680},[525],[47,29682,29684,29704],{"className":29683,"style":25158},[529],[47,29685,29686,29689],{"style":26288},[47,29687],{"className":29688,"style":929},[537],[47,29690,29692],{"className":29691},[542,543,544,545],[47,29693,29695,29698,29701],{"className":29694},[508,545],[47,29696,1166],{"className":29697},[508,581,545],[47,29699,6147],{"className":29700},[942,545],[47,29702,3192],{"className":29703},[508,545],[47,29705,29706,29709],{"style":16788},[47,29707],{"className":29708,"style":929},[537],[47,29710,29712],{"className":29711},[542,543,544,545],[47,29713,1061],{"className":29714,"style":1091},[508,581,545],[47,29716,616],{"className":29717},[615],[47,29719,29721],{"className":29720},[525],[47,29722,29724],{"className":29723,"style":26672},[529],[47,29725],{},[47,29727,1201],{"className":29728},[702],[47,29730],{"className":29731,"style":2259},[888],[47,29733,6147],{"className":29734},[942],[47,29736],{"className":29737,"style":2259},[888],[47,29739,23606],{"className":29740},[1349],[47,29742],{"className":29743,"style":1303},[888],[47,29745,1201],{"className":29746},[702],[47,29748,29749,29752],{"style":28601},[47,29750],{"className":29751,"style":28471},[537],[47,29753,29755,29758,29761,29764,29767,29770,29773,29776,29779,29782,29785,29788,29791,29842,29845,29848,29851,29854,29857,29860,29863,29866,29869,29898,29901,29904,29907,29910,29913,29916,29919,29922,29925,29928,29988,29991,29994,29997,30000,30003,30006,30009,30012,30041,30044,30047,30050,30053,30056,30059,30062,30065,30068,30071,30074],{"className":29754},[508],[47,29756],{"className":29757},[508],[47,29759],{"className":29760,"style":889},[888],[47,29762,82],{"className":29763},[893],[47,29765],{"className":29766,"style":889},[888],[47,29768,1187],{"className":29769},[512],[47,29771,472],{"className":29772},[508],[47,29774],{"className":29775,"style":2259},[888],[47,29777,18727],{"className":29778},[942],[47,29780],{"className":29781,"style":2259},[888],[47,29783,27822],{"className":29784},[508,581],[47,29786,1201],{"className":29787},[702],[47,29789,1187],{"className":29790},[512],[47,29792,29794,29797],{"className":29793},[508],[47,29795,25839],{"className":29796,"style":25860},[508,581],[47,29798,29800],{"className":29799},[585],[47,29801,29803,29834],{"className":29802},[520,521],[47,29804,29806,29831],{"className":29805},[525],[47,29807,29809,29820],{"className":29808,"style":25158},[529],[47,29810,29811,29814],{"style":26288},[47,29812],{"className":29813,"style":929},[537],[47,29815,29817],{"className":29816},[542,543,544,545],[47,29818,1166],{"className":29819},[508,581,545],[47,29821,29822,29825],{"style":16788},[47,29823],{"className":29824,"style":929},[537],[47,29826,29828],{"className":29827},[542,543,544,545],[47,29829,1061],{"className":29830,"style":1091},[508,581,545],[47,29832,616],{"className":29833},[615],[47,29835,29837],{"className":29836},[525],[47,29838,29840],{"className":29839,"style":17703},[529],[47,29841],{},[47,29843,1187],{"className":29844},[512],[47,29846,472],{"className":29847},[508],[47,29849],{"className":29850,"style":2259},[888],[47,29852,6147],{"className":29853},[942],[47,29855],{"className":29856,"style":2259},[888],[47,29858,27822],{"className":29859},[508,581],[47,29861],{"className":29862,"style":2259},[888],[47,29864,6147],{"className":29865},[942],[47,29867],{"className":29868,"style":2259},[888],[47,29870,29872,29875],{"className":29871},[508],[47,29873,27822],{"className":29874},[508,581],[47,29876,29878],{"className":29877},[585],[47,29879,29881],{"className":29880},[520],[47,29882,29884],{"className":29883},[525],[47,29885,29887],{"className":29886,"style":23672},[529],[47,29888,29889,29892],{"style":16788},[47,29890],{"className":29891,"style":929},[537],[47,29893,29895],{"className":29894},[542,543,544,545],[47,29896,3192],{"className":29897},[508,545],[47,29899],{"className":29900,"style":2259},[888],[47,29902,6147],{"className":29903},[942],[47,29905],{"className":29906,"style":2259},[888],[47,29908,23606],{"className":29909},[1349],[47,29911],{"className":29912,"style":1303},[888],[47,29914,1201],{"className":29915},[702],[47,29917],{"className":29918,"style":2259},[888],[47,29920,6147],{"className":29921},[942],[47,29923],{"className":29924,"style":2259},[888],[47,29926,19357],{"className":29927,"style":19604},[508,581],[47,29929,29931,29934],{"className":29930},[508],[47,29932,25839],{"className":29933,"style":25860},[508,581],[47,29935,29937],{"className":29936},[585],[47,29938,29940,29980],{"className":29939},[520,521],[47,29941,29943,29977],{"className":29942},[525],[47,29944,29946,29966],{"className":29945,"style":25158},[529],[47,29947,29948,29951],{"style":26288},[47,29949],{"className":29950,"style":929},[537],[47,29952,29954],{"className":29953},[542,543,544,545],[47,29955,29957,29960,29963],{"className":29956},[508,545],[47,29958,1166],{"className":29959},[508,581,545],[47,29961,6147],{"className":29962},[942,545],[47,29964,472],{"className":29965},[508,545],[47,29967,29968,29971],{"style":16788},[47,29969],{"className":29970,"style":929},[537],[47,29972,29974],{"className":29973},[542,543,544,545],[47,29975,1061],{"className":29976,"style":1091},[508,581,545],[47,29978,616],{"className":29979},[615],[47,29981,29983],{"className":29982},[525],[47,29984,29986],{"className":29985,"style":26672},[529],[47,29987],{},[47,29989],{"className":29990,"style":2259},[888],[47,29992,23411],{"className":29993},[942],[47,29995],{"className":29996,"style":2259},[888],[47,29998,1187],{"className":29999},[512],[47,30001,27822],{"className":30002},[508,581],[47,30004],{"className":30005,"style":2259},[888],[47,30007,6147],{"className":30008},[942],[47,30010],{"className":30011,"style":2259},[888],[47,30013,30015,30018],{"className":30014},[508],[47,30016,27822],{"className":30017},[508,581],[47,30019,30021],{"className":30020},[585],[47,30022,30024],{"className":30023},[520],[47,30025,30027],{"className":30026},[525],[47,30028,30030],{"className":30029,"style":23672},[529],[47,30031,30032,30035],{"style":16788},[47,30033],{"className":30034,"style":929},[537],[47,30036,30038],{"className":30037},[542,543,544,545],[47,30039,3192],{"className":30040},[508,545],[47,30042],{"className":30043,"style":2259},[888],[47,30045,6147],{"className":30046},[942],[47,30048],{"className":30049,"style":2259},[888],[47,30051,23606],{"className":30052},[1349],[47,30054],{"className":30055,"style":1303},[888],[47,30057,1201],{"className":30058},[702],[47,30060],{"className":30061,"style":2259},[888],[47,30063,6147],{"className":30064},[942],[47,30066],{"className":30067,"style":2259},[888],[47,30069,23606],{"className":30070},[1349],[47,30072],{"className":30073,"style":1303},[888],[47,30075,1201],{"className":30076},[702],[47,30078,30079,30082],{"style":28610},[47,30080],{"className":30081,"style":28471},[537],[47,30083,30085,30088,30091,30094,30097,30100,30103,30106,30109,30112,30115,30118,30121],{"className":30084},[508],[47,30086],{"className":30087},[508],[47,30089],{"className":30090,"style":889},[888],[47,30092,82],{"className":30093},[893],[47,30095],{"className":30096,"style":889},[888],[47,30098,1187],{"className":30099},[512],[47,30101,472],{"className":30102},[508],[47,30104],{"className":30105,"style":2259},[888],[47,30107,18727],{"className":30108},[942],[47,30110],{"className":30111,"style":2259},[888],[47,30113,27822],{"className":30114},[508,581],[47,30116,1201],{"className":30117},[702],[47,30119],{"className":30120,"style":1303},[888],[47,30122,30124,30130,30181,30256,30259,30262,30265,30268,30328,30403,30406,30409,30412,30415,30418],{"className":30123},[1349],[47,30125,30127],{"className":30126,"style":1354},[512,1353],[47,30128,1187],{"className":30129},[1358,544],[47,30131,30133,30136],{"className":30132},[508],[47,30134,25839],{"className":30135,"style":25860},[508,581],[47,30137,30139],{"className":30138},[585],[47,30140,30142,30173],{"className":30141},[520,521],[47,30143,30145,30170],{"className":30144},[525],[47,30146,30148,30159],{"className":30147,"style":25158},[529],[47,30149,30150,30153],{"style":26288},[47,30151],{"className":30152,"style":929},[537],[47,30154,30156],{"className":30155},[542,543,544,545],[47,30157,1166],{"className":30158},[508,581,545],[47,30160,30161,30164],{"style":16788},[47,30162],{"className":30163,"style":929},[537],[47,30165,30167],{"className":30166},[542,543,544,545],[47,30168,1061],{"className":30169,"style":1091},[508,581,545],[47,30171,616],{"className":30172},[615],[47,30174,30176],{"className":30175},[525],[47,30177,30179],{"className":30178,"style":17703},[529],[47,30180],{},[47,30182,30184,30187,30253],{"className":30183},[508],[47,30185],{"className":30186},[512,513],[47,30188,30190],{"className":30189},[467],[47,30191,30193,30244],{"className":30192},[520,521],[47,30194,30196,30241],{"className":30195},[525],[47,30197,30199,30222,30230],{"className":30198,"style":16641},[529],[47,30200,30201,30204],{"style":16181},[47,30202],{"className":30203,"style":538},[537],[47,30205,30207,30210,30213,30216,30219],{"className":30206},[508],[47,30208,472],{"className":30209},[508],[47,30211],{"className":30212,"style":2259},[888],[47,30214,18727],{"className":30215},[942],[47,30217],{"className":30218,"style":2259},[888],[47,30220,27822],{"className":30221},[508,581],[47,30223,30224,30227],{"style":663},[47,30225],{"className":30226,"style":538},[537],[47,30228],{"className":30229,"style":671},[670],[47,30231,30232,30235],{"style":16247},[47,30233],{"className":30234,"style":538},[537],[47,30236,30238],{"className":30237},[508],[47,30239,472],{"className":30240},[508],[47,30242,616],{"className":30243},[615],[47,30245,30247],{"className":30246},[525],[47,30248,30251],{"className":30249,"style":30250},[529],"height:0.7693em;",[47,30252],{},[47,30254],{"className":30255},[702,513],[47,30257],{"className":30258,"style":2259},[888],[47,30260,6147],{"className":30261},[942],[47,30263],{"className":30264,"style":2259},[888],[47,30266,19357],{"className":30267,"style":19604},[508,581],[47,30269,30271,30274],{"className":30270},[508],[47,30272,25839],{"className":30273,"style":25860},[508,581],[47,30275,30277],{"className":30276},[585],[47,30278,30280,30320],{"className":30279},[520,521],[47,30281,30283,30317],{"className":30282},[525],[47,30284,30286,30306],{"className":30285,"style":25158},[529],[47,30287,30288,30291],{"style":26288},[47,30289],{"className":30290,"style":929},[537],[47,30292,30294],{"className":30293},[542,543,544,545],[47,30295,30297,30300,30303],{"className":30296},[508,545],[47,30298,1166],{"className":30299},[508,581,545],[47,30301,6147],{"className":30302},[942,545],[47,30304,472],{"className":30305},[508,545],[47,30307,30308,30311],{"style":16788},[47,30309],{"className":30310,"style":929},[537],[47,30312,30314],{"className":30313},[542,543,544,545],[47,30315,1061],{"className":30316,"style":1091},[508,581,545],[47,30318,616],{"className":30319},[615],[47,30321,30323],{"className":30322},[525],[47,30324,30326],{"className":30325,"style":26672},[529],[47,30327],{},[47,30329,30331,30334,30400],{"className":30330},[508],[47,30332],{"className":30333},[512,513],[47,30335,30337],{"className":30336},[467],[47,30338,30340,30392],{"className":30339},[520,521],[47,30341,30343,30389],{"className":30342},[525],[47,30344,30347,30370,30378],{"className":30345,"style":30346},[529],"height:1.3714em;",[47,30348,30349,30352],{"style":16181},[47,30350],{"className":30351,"style":538},[537],[47,30353,30355,30358,30361,30364,30367],{"className":30354},[508],[47,30356,472],{"className":30357},[508],[47,30359],{"className":30360,"style":2259},[888],[47,30362,18727],{"className":30363},[942],[47,30365],{"className":30366,"style":2259},[888],[47,30368,27822],{"className":30369},[508,581],[47,30371,30372,30375],{"style":663},[47,30373],{"className":30374,"style":538},[537],[47,30376],{"className":30377,"style":671},[670],[47,30379,30380,30383],{"style":16247},[47,30381],{"className":30382,"style":538},[537],[47,30384,30386],{"className":30385},[508],[47,30387,27822],{"className":30388},[508,581],[47,30390,616],{"className":30391},[615],[47,30393,30395],{"className":30394},[525],[47,30396,30398],{"className":30397,"style":30250},[529],[47,30399],{},[47,30401],{"className":30402},[702,513],[47,30404],{"className":30405,"style":2259},[888],[47,30407,6147],{"className":30408},[942],[47,30410],{"className":30411,"style":2259},[888],[47,30413,23606],{"className":30414},[1349],[47,30416],{"className":30417,"style":1303},[888],[47,30419,30421],{"className":30420,"style":1354},[702,1353],[47,30422,1201],{"className":30423},[1358,544],[47,30425,30426,30429],{"style":28619},[47,30427],{"className":30428,"style":28471},[537],[47,30430,30432,30435,30438,30441,30444,30511,30514,30517,30520,30550],{"className":30431},[508],[47,30433],{"className":30434},[508],[47,30436],{"className":30437,"style":889},[888],[47,30439,82],{"className":30440},[893],[47,30442],{"className":30443,"style":889},[888],[47,30445,30447],{"className":30446},[1338,14600],[47,30448,30450,30502],{"className":30449},[520,521],[47,30451,30453,30499],{"className":30452},[525],[47,30454,30457,30478,30488],{"className":30455,"style":30456},[529],"height:1.6514em;",[47,30458,30460,30463],{"style":30459},"top:-1.8479em;margin-left:0em;",[47,30461],{"className":30462,"style":15794},[537],[47,30464,30466],{"className":30465},[542,543,544,545],[47,30467,30469,30472,30475],{"className":30468},[508,545],[47,30470,7727],{"className":30471},[508,581,545],[47,30473,82],{"className":30474},[893,545],[47,30476,2141],{"className":30477},[508,545],[47,30479,30480,30483],{"style":15803},[47,30481],{"className":30482,"style":15794},[537],[47,30484,30485],{},[47,30486,2213],{"className":30487},[1338,2297,15812],[47,30489,30490,30493],{"style":16736},[47,30491],{"className":30492,"style":15794},[537],[47,30494,30496],{"className":30495},[542,543,544,545],[47,30497,28406],{"className":30498},[508,545],[47,30500,616],{"className":30501},[615],[47,30503,30505],{"className":30504},[525],[47,30506,30509],{"className":30507,"style":30508},[529],"height:1.3021em;",[47,30510],{},[47,30512,1187],{"className":30513},[512],[47,30515,19357],{"className":30516,"style":19604},[508,581],[47,30518,27822],{"className":30519},[508,581],[47,30521,30523,30526],{"className":30522},[702],[47,30524,1201],{"className":30525},[702],[47,30527,30529],{"className":30528},[585],[47,30530,30532],{"className":30531},[520],[47,30533,30535],{"className":30534},[525],[47,30536,30539],{"className":30537,"style":30538},[529],"height:0.8991em;",[47,30540,30541,30544],{"style":16788},[47,30542],{"className":30543,"style":929},[537],[47,30545,30547],{"className":30546},[542,543,544,545],[47,30548,7727],{"className":30549},[508,581,545],[47,30551,30553,30556],{"className":30552},[508],[47,30554,25839],{"className":30555,"style":25860},[508,581],[47,30557,30559],{"className":30558},[585],[47,30560,30562,30602],{"className":30561},[520,521],[47,30563,30565,30599],{"className":30564},[525],[47,30566,30568,30588],{"className":30567,"style":25158},[529],[47,30569,30570,30573],{"style":26288},[47,30571],{"className":30572,"style":929},[537],[47,30574,30576],{"className":30575},[542,543,544,545],[47,30577,30579,30582,30585],{"className":30578},[508,545],[47,30580,1166],{"className":30581},[508,581,545],[47,30583,6147],{"className":30584},[942,545],[47,30586,7727],{"className":30587},[508,581,545],[47,30589,30590,30593],{"style":16788},[47,30591],{"className":30592,"style":929},[537],[47,30594,30596],{"className":30595},[542,543,544,545],[47,30597,1061],{"className":30598,"style":1091},[508,581,545],[47,30600,616],{"className":30601},[615],[47,30603,30605],{"className":30604},[525],[47,30606,30608],{"className":30607,"style":26672},[529],[47,30609],{},[47,30611,616],{"className":30612},[615],[47,30614,30616],{"className":30615},[525],[47,30617,30619],{"className":30618,"style":28635},[529],[47,30620],{},[14,30622,30623],{},"整理后发现其中有等比数列，利用等比数列求和得到最终的 GAE 优势函数的表达式。",[14,30625,30626],{},"GAE 优势函数平衡了采样不同步带来的方差和偏差的问题。",[14,30628,30629],{},"现在我们来看上面得之不易的三个表达式：",[47,30631,30633],{"className":30632},[14712],[47,30634,30636,30892],{"className":30635},[451],[47,30637,30639],{"className":30638},[455],[457,30640,30641],{"xmlns":459,"display":9848},[461,30642,30643,30889],{},[464,30644,30645,30653,30655,30661,30663,30665,30671,30673,30685,30687,30689,30695,30697,30703,30705,30707,30721,30723,30729,30731,30733,30735,30737,30751,30753,30755,30757,30763,30777,30779,30785,30799,30817,30831,30833,30841,30843,30851,30853,30855,30857,30859,30861,30867,30869,30877,30879,30887],{},[2209,30646,30647,30649,30651],{},[480,30648,25839],{},[480,30650,1166],{},[480,30652,1061],{},[850,30654,82],{},[477,30656,30657,30659],{},[480,30658,8196],{},[480,30660,1166],{},[850,30662,6147],{},[480,30664,19357],{},[477,30666,30667,30669],{},[480,30668,1061],{},[480,30670,14413],{},[850,30672,1187],{"stretchy":1186},[477,30674,30675,30677],{},[480,30676,13164],{},[464,30678,30679,30681,30683],{},[480,30680,1166],{},[850,30682,6147],{},[470,30684,472],{},[850,30686,1201],{"stretchy":1186},[850,30688,18727],{},[477,30690,30691,30693],{},[480,30692,1061],{},[480,30694,14413],{},[850,30696,1187],{"stretchy":1186},[477,30698,30699,30701],{},[480,30700,13164],{},[480,30702,1166],{},[850,30704,1201],{"stretchy":1186},[888,30706],{"linebreak":22015},[2209,30708,30709,30711,30713],{},[480,30710,1163],{},[480,30712,14413],{},[464,30714,30715,30717,30719],{},[480,30716,27872],{},[480,30718,1163],{},[480,30720,5977],{},[850,30722,1187],{"stretchy":1186},[477,30724,30725,30727],{},[480,30726,13164],{},[480,30728,1166],{},[850,30730,1192],{"separator":495},[480,30732,285],{},[850,30734,1201],{"stretchy":1186},[850,30736,82],{},[15045,30738,30739,30741,30749],{},[850,30740,2213],{},[464,30742,30743,30745,30747],{},[480,30744,7727],{},[850,30746,82],{},[470,30748,2141],{},[480,30750,28406],{"mathvariant":1206},[850,30752,1187],{"stretchy":1186},[480,30754,19357],{},[480,30756,27822],{},[854,30758,30759,30761],{},[850,30760,1201],{"stretchy":1186},[480,30762,7727],{},[2209,30764,30765,30767,30775],{},[480,30766,25839],{},[464,30768,30769,30771,30773],{},[480,30770,1166],{},[850,30772,6147],{},[480,30774,7727],{},[480,30776,1061],{},[888,30778],{"linebreak":22015},[467,30780,30781,30783],{},[470,30782,472],{},[480,30784,7692],{},[15045,30786,30787,30789,30797],{},[850,30788,2213],{},[464,30790,30791,30793,30795],{},[480,30792,724],{},[850,30794,82],{},[470,30796,472],{},[480,30798,7692],{},[15045,30800,30801,30803,30811],{},[850,30802,2213],{},[464,30804,30805,30807,30809],{},[480,30806,1166],{},[850,30808,82],{},[470,30810,472],{},[477,30812,30813,30815],{},[480,30814,13604],{},[480,30816,724],{},[2209,30818,30819,30821,30823],{},[480,30820,1163],{},[480,30822,14413],{},[464,30824,30825,30827,30829],{},[480,30826,27872],{},[480,30828,1163],{},[480,30830,5977],{},[850,30832,1187],{"stretchy":1186},[2209,30834,30835,30837,30839],{},[480,30836,13164],{},[480,30838,724],{},[480,30840,1166],{},[850,30842,1192],{"separator":495},[2209,30844,30845,30847,30849],{},[480,30846,285],{},[480,30848,724],{},[480,30850,1166],{},[850,30852,1201],{"stretchy":1186},[3647,30854,27957],{},[480,30856,14742],{"mathvariant":1206},[480,30858,15167],{},[850,30860,1210],{},[477,30862,30863,30865],{},[480,30864,5974],{},[480,30866,14413],{},[850,30868,1187],{"stretchy":1186},[2209,30870,30871,30873,30875],{},[480,30872,285],{},[480,30874,724],{},[480,30876,1166],{},[480,30878,13303],{"mathvariant":1206},[2209,30880,30881,30883,30885],{},[480,30882,13164],{},[480,30884,724],{},[480,30886,1166],{},[850,30888,1201],{"stretchy":1186},[487,30890,30891],{"encoding":489},"\\delta_t^V = r_t + \\gamma V_\\theta(s_{t+1}) - V_\\theta(s_t) \\\\\nA_\\theta^{GAE}(s_t, a) = \\sum_{b=0}^{\\infty} (\\gamma\\lambda)^b \\delta_{t+b}^V \\\\\n\\frac{1}{N}\\sum_{n=1}^N\\sum_{t=1}^{T_n} A_\\theta^{GAE}(s_n^t, a_n^t)\\, \\nabla \\log P_\\theta(a_n^t|s_n^t)",[47,30893,30895,30961,31016,31129,31221,31224,31354,31526,31529],{"className":30894,"ariaHidden":495},[494],[47,30896,30898,30901,30952,30955,30958],{"className":30897},[499],[47,30899],{"className":30900,"style":26267},[503],[47,30902,30904,30907],{"className":30903},[508],[47,30905,25839],{"className":30906,"style":25860},[508,581],[47,30908,30910],{"className":30909},[585],[47,30911,30913,30944],{"className":30912},[520,521],[47,30914,30916,30941],{"className":30915},[525],[47,30917,30919,30930],{"className":30918,"style":25158},[529],[47,30920,30921,30924],{"style":26288},[47,30922],{"className":30923,"style":929},[537],[47,30925,30927],{"className":30926},[542,543,544,545],[47,30928,1166],{"className":30929},[508,581,545],[47,30931,30932,30935],{"style":16788},[47,30933],{"className":30934,"style":929},[537],[47,30936,30938],{"className":30937},[542,543,544,545],[47,30939,1061],{"className":30940,"style":1091},[508,581,545],[47,30942,616],{"className":30943},[615],[47,30945,30947],{"className":30946},[525],[47,30948,30950],{"className":30949,"style":17703},[529],[47,30951],{},[47,30953],{"className":30954,"style":889},[888],[47,30956,82],{"className":30957},[893],[47,30959],{"className":30960,"style":889},[888],[47,30962,30964,30967,31007,31010,31013],{"className":30963},[499],[47,30965],{"className":30966,"style":22274},[503],[47,30968,30970,30973],{"className":30969},[508],[47,30971,8196],{"className":30972,"style":3580},[508,581],[47,30974,30976],{"className":30975},[585],[47,30977,30979,30999],{"className":30978},[520,521],[47,30980,30982,30996],{"className":30981},[525],[47,30983,30985],{"className":30984,"style":13094},[529],[47,30986,30987,30990],{"style":13467},[47,30988],{"className":30989,"style":929},[537],[47,30991,30993],{"className":30992},[542,543,544,545],[47,30994,1166],{"className":30995},[508,581,545],[47,30997,616],{"className":30998},[615],[47,31000,31002],{"className":31001},[525],[47,31003,31005],{"className":31004,"style":1657},[529],[47,31006],{},[47,31008],{"className":31009,"style":2259},[888],[47,31011,6147],{"className":31012},[942],[47,31014],{"className":31015,"style":2259},[888],[47,31017,31019,31022,31025,31065,31068,31117,31120,31123,31126],{"className":31018},[499],[47,31020],{"className":31021,"style":1264},[503],[47,31023,19357],{"className":31024,"style":19604},[508,581],[47,31026,31028,31031],{"className":31027},[508],[47,31029,1061],{"className":31030,"style":1091},[508,581],[47,31032,31034],{"className":31033},[585],[47,31035,31037,31057],{"className":31036},[520,521],[47,31038,31040,31054],{"className":31039},[525],[47,31041,31043],{"className":31042,"style":1635},[529],[47,31044,31045,31048],{"style":20867},[47,31046],{"className":31047,"style":929},[537],[47,31049,31051],{"className":31050},[542,543,544,545],[47,31052,14413],{"className":31053,"style":3580},[508,581,545],[47,31055,616],{"className":31056},[615],[47,31058,31060],{"className":31059},[525],[47,31061,31063],{"className":31062,"style":1657},[529],[47,31064],{},[47,31066,1187],{"className":31067},[512],[47,31069,31071,31074],{"className":31070},[508],[47,31072,13164],{"className":31073},[508,581],[47,31075,31077],{"className":31076},[585],[47,31078,31080,31109],{"className":31079},[520,521],[47,31081,31083,31106],{"className":31082},[525],[47,31084,31086],{"className":31085,"style":3461},[529],[47,31087,31088,31091],{"style":1638},[47,31089],{"className":31090,"style":929},[537],[47,31092,31094],{"className":31093},[542,543,544,545],[47,31095,31097,31100,31103],{"className":31096},[508,545],[47,31098,1166],{"className":31099},[508,581,545],[47,31101,6147],{"className":31102},[942,545],[47,31104,472],{"className":31105},[508,545],[47,31107,616],{"className":31108},[615],[47,31110,31112],{"className":31111},[525],[47,31113,31115],{"className":31114,"style":7308},[529],[47,31116],{},[47,31118,1201],{"className":31119},[702],[47,31121],{"className":31122,"style":2259},[888],[47,31124,18727],{"className":31125},[942],[47,31127],{"className":31128,"style":2259},[888],[47,31130,31132,31135,31175,31178,31218],{"className":31131},[499],[47,31133],{"className":31134,"style":1264},[503],[47,31136,31138,31141],{"className":31137},[508],[47,31139,1061],{"className":31140,"style":1091},[508,581],[47,31142,31144],{"className":31143},[585],[47,31145,31147,31167],{"className":31146},[520,521],[47,31148,31150,31164],{"className":31149},[525],[47,31151,31153],{"className":31152,"style":1635},[529],[47,31154,31155,31158],{"style":20867},[47,31156],{"className":31157,"style":929},[537],[47,31159,31161],{"className":31160},[542,543,544,545],[47,31162,14413],{"className":31163,"style":3580},[508,581,545],[47,31165,616],{"className":31166},[615],[47,31168,31170],{"className":31169},[525],[47,31171,31173],{"className":31172,"style":1657},[529],[47,31174],{},[47,31176,1187],{"className":31177},[512],[47,31179,31181,31184],{"className":31180},[508],[47,31182,13164],{"className":31183},[508,581],[47,31185,31187],{"className":31186},[585],[47,31188,31190,31210],{"className":31189},[520,521],[47,31191,31193,31207],{"className":31192},[525],[47,31194,31196],{"className":31195,"style":13094},[529],[47,31197,31198,31201],{"style":1638},[47,31199],{"className":31200,"style":929},[537],[47,31202,31204],{"className":31203},[542,543,544,545],[47,31205,1166],{"className":31206},[508,581,545],[47,31208,616],{"className":31209},[615],[47,31211,31213],{"className":31212},[525],[47,31214,31216],{"className":31215,"style":1657},[529],[47,31217],{},[47,31219,1201],{"className":31220},[702],[47,31222],{"className":31223},[888,22015],[47,31225,31227,31230,31290,31293,31333,31336,31339,31342,31345,31348,31351],{"className":31226},[499],[47,31228],{"className":31229,"style":25139},[503],[47,31231,31233,31236],{"className":31232},[508],[47,31234,1163],{"className":31235},[508,581],[47,31237,31239],{"className":31238},[585],[47,31240,31242,31282],{"className":31241},[520,521],[47,31243,31245,31279],{"className":31244},[525],[47,31246,31248,31259],{"className":31247,"style":25158},[529],[47,31249,31250,31253],{"style":17673},[47,31251],{"className":31252,"style":929},[537],[47,31254,31256],{"className":31255},[542,543,544,545],[47,31257,14413],{"className":31258,"style":3580},[508,581,545],[47,31260,31261,31264],{"style":16788},[47,31262],{"className":31263,"style":929},[537],[47,31265,31267],{"className":31266},[542,543,544,545],[47,31268,31270,31273,31276],{"className":31269},[508,545],[47,31271,27872],{"className":31272},[508,581,545],[47,31274,1163],{"className":31275},[508,581,545],[47,31277,5977],{"className":31278,"style":6420},[508,581,545],[47,31280,616],{"className":31281},[615],[47,31283,31285],{"className":31284},[525],[47,31286,31288],{"className":31287,"style":17703},[529],[47,31289],{},[47,31291,1187],{"className":31292},[512],[47,31294,31296,31299],{"className":31295},[508],[47,31297,13164],{"className":31298},[508,581],[47,31300,31302],{"className":31301},[585],[47,31303,31305,31325],{"className":31304},[520,521],[47,31306,31308,31322],{"className":31307},[525],[47,31309,31311],{"className":31310,"style":13094},[529],[47,31312,31313,31316],{"style":1638},[47,31314],{"className":31315,"style":929},[537],[47,31317,31319],{"className":31318},[542,543,544,545],[47,31320,1166],{"className":31321},[508,581,545],[47,31323,616],{"className":31324},[615],[47,31326,31328],{"className":31327},[525],[47,31329,31331],{"className":31330,"style":1657},[529],[47,31332],{},[47,31334,1192],{"className":31335},[1299],[47,31337],{"className":31338,"style":1303},[888],[47,31340,285],{"className":31341},[508,581],[47,31343,1201],{"className":31344},[702],[47,31346],{"className":31347,"style":889},[888],[47,31349,82],{"className":31350},[893],[47,31352],{"className":31353,"style":889},[888],[47,31355,31357,31361,31428,31431,31434,31437,31466],{"className":31356},[499],[47,31358],{"className":31359,"style":31360},[503],"height:2.9535em;vertical-align:-1.3021em;",[47,31362,31364],{"className":31363},[1338,14600],[47,31365,31367,31420],{"className":31366},[520,521],[47,31368,31370,31417],{"className":31369},[525],[47,31371,31373,31393,31403],{"className":31372,"style":30456},[529],[47,31374,31375,31378],{"style":30459},[47,31376],{"className":31377,"style":15794},[537],[47,31379,31381],{"className":31380},[542,543,544,545],[47,31382,31384,31387,31390],{"className":31383},[508,545],[47,31385,7727],{"className":31386},[508,581,545],[47,31388,82],{"className":31389},[893,545],[47,31391,2141],{"className":31392},[508,545],[47,31394,31395,31398],{"style":15803},[47,31396],{"className":31397,"style":15794},[537],[47,31399,31400],{},[47,31401,2213],{"className":31402},[1338,2297,15812],[47,31404,31405,31408],{"style":16736},[47,31406],{"className":31407,"style":15794},[537],[47,31409,31411],{"className":31410},[542,543,544,545],[47,31412,31414],{"className":31413},[508,545],[47,31415,28406],{"className":31416},[508,545],[47,31418,616],{"className":31419},[615],[47,31421,31423],{"className":31422},[525],[47,31424,31426],{"className":31425,"style":30508},[529],[47,31427],{},[47,31429,1187],{"className":31430},[512],[47,31432,19357],{"className":31433,"style":19604},[508,581],[47,31435,27822],{"className":31436},[508,581],[47,31438,31440,31443],{"className":31439},[702],[47,31441,1201],{"className":31442},[702],[47,31444,31446],{"className":31445},[585],[47,31447,31449],{"className":31448},[520],[47,31450,31452],{"className":31451},[525],[47,31453,31455],{"className":31454,"style":30538},[529],[47,31456,31457,31460],{"style":16788},[47,31458],{"className":31459,"style":929},[537],[47,31461,31463],{"className":31462},[542,543,544,545],[47,31464,7727],{"className":31465},[508,581,545],[47,31467,31469,31472],{"className":31468},[508],[47,31470,25839],{"className":31471,"style":25860},[508,581],[47,31473,31475],{"className":31474},[585],[47,31476,31478,31518],{"className":31477},[520,521],[47,31479,31481,31515],{"className":31480},[525],[47,31482,31484,31504],{"className":31483,"style":25158},[529],[47,31485,31486,31489],{"style":26288},[47,31487],{"className":31488,"style":929},[537],[47,31490,31492],{"className":31491},[542,543,544,545],[47,31493,31495,31498,31501],{"className":31494},[508,545],[47,31496,1166],{"className":31497},[508,581,545],[47,31499,6147],{"className":31500},[942,545],[47,31502,7727],{"className":31503},[508,581,545],[47,31505,31506,31509],{"style":16788},[47,31507],{"className":31508,"style":929},[537],[47,31510,31512],{"className":31511},[542,543,544,545],[47,31513,1061],{"className":31514,"style":1091},[508,581,545],[47,31516,616],{"className":31517},[615],[47,31519,31521],{"className":31520},[525],[47,31522,31524],{"className":31523,"style":26672},[529],[47,31525],{},[47,31527],{"className":31528},[888,22015],[47,31530,31532,31536,31598,31601,31665,31668,31772,31775,31835,31838,31889,31892,31895,31946,31949,31952,31955,31958,31963,31966,32006,32009,32060,32063,32114],{"className":31531},[499],[47,31533],{"className":31534,"style":31535},[503],"height:3.1065em;vertical-align:-1.2671em;",[47,31537,31539,31542,31595],{"className":31538},[508],[47,31540],{"className":31541},[512,513],[47,31543,31545],{"className":31544},[467],[47,31546,31548,31587],{"className":31547},[520,521],[47,31549,31551,31584],{"className":31550},[525],[47,31552,31554,31565,31573],{"className":31553,"style":16641},[529],[47,31555,31556,31559],{"style":16181},[47,31557],{"className":31558,"style":538},[537],[47,31560,31562],{"className":31561},[508],[47,31563,7692],{"className":31564,"style":7767},[508,581],[47,31566,31567,31570],{"style":663},[47,31568],{"className":31569,"style":538},[537],[47,31571],{"className":31572,"style":671},[670],[47,31574,31575,31578],{"style":16247},[47,31576],{"className":31577,"style":538},[537],[47,31579,31581],{"className":31580},[508],[47,31582,472],{"className":31583},[508],[47,31585,616],{"className":31586},[615],[47,31588,31590],{"className":31589},[525],[47,31591,31593],{"className":31592,"style":16681},[529],[47,31594],{},[47,31596],{"className":31597},[702,513],[47,31599],{"className":31600,"style":1303},[888],[47,31602,31604],{"className":31603},[1338,14600],[47,31605,31607,31657],{"className":31606},[520,521],[47,31608,31610,31654],{"className":31609},[525],[47,31611,31613,31633,31643],{"className":31612,"style":16702},[529],[47,31614,31615,31618],{"style":16705},[47,31616],{"className":31617,"style":15794},[537],[47,31619,31621],{"className":31620},[542,543,544,545],[47,31622,31624,31627,31630],{"className":31623},[508,545],[47,31625,724],{"className":31626},[508,581,545],[47,31628,82],{"className":31629},[893,545],[47,31631,472],{"className":31632},[508,545],[47,31634,31635,31638],{"style":15803},[47,31636],{"className":31637,"style":15794},[537],[47,31639,31640],{},[47,31641,2213],{"className":31642},[1338,2297,15812],[47,31644,31645,31648],{"style":16736},[47,31646],{"className":31647,"style":15794},[537],[47,31649,31651],{"className":31650},[542,543,544,545],[47,31652,7692],{"className":31653,"style":7767},[508,581,545],[47,31655,616],{"className":31656},[615],[47,31658,31660],{"className":31659},[525],[47,31661,31663],{"className":31662,"style":16755},[529],[47,31664],{},[47,31666],{"className":31667,"style":1303},[888],[47,31669,31671],{"className":31670},[1338,14600],[47,31672,31674,31764],{"className":31673},[520,521],[47,31675,31677,31761],{"className":31676},[525],[47,31678,31680,31700,31710],{"className":31679,"style":17511},[529],[47,31681,31682,31685],{"style":16705},[47,31683],{"className":31684,"style":15794},[537],[47,31686,31688],{"className":31687},[542,543,544,545],[47,31689,31691,31694,31697],{"className":31690},[508,545],[47,31692,1166],{"className":31693},[508,581,545],[47,31695,82],{"className":31696},[893,545],[47,31698,472],{"className":31699},[508,545],[47,31701,31702,31705],{"style":15803},[47,31703],{"className":31704,"style":15794},[537],[47,31706,31707],{},[47,31708,2213],{"className":31709},[1338,2297,15812],[47,31711,31712,31715],{"style":17544},[47,31713],{"className":31714,"style":15794},[537],[47,31716,31718],{"className":31717},[542,543,544,545],[47,31719,31721],{"className":31720},[508,545],[47,31722,31724,31727],{"className":31723},[508,545],[47,31725,13604],{"className":31726,"style":3555},[508,581,545],[47,31728,31730],{"className":31729},[585],[47,31731,31733,31753],{"className":31732},[520,521],[47,31734,31736,31750],{"className":31735},[525],[47,31737,31739],{"className":31738,"style":4672},[529],[47,31740,31741,31744],{"style":17574},[47,31742],{"className":31743,"style":602},[537],[47,31745,31747],{"className":31746},[542,606,607,545],[47,31748,724],{"className":31749},[508,581,545],[47,31751,616],{"className":31752},[615],[47,31754,31756],{"className":31755},[525],[47,31757,31759],{"className":31758,"style":4694},[529],[47,31760],{},[47,31762,616],{"className":31763},[615],[47,31765,31767],{"className":31766},[525],[47,31768,31770],{"className":31769,"style":16755},[529],[47,31771],{},[47,31773],{"className":31774,"style":1303},[888],[47,31776,31778,31781],{"className":31777},[508],[47,31779,1163],{"className":31780},[508,581],[47,31782,31784],{"className":31783},[585],[47,31785,31787,31827],{"className":31786},[520,521],[47,31788,31790,31824],{"className":31789},[525],[47,31791,31793,31804],{"className":31792,"style":25158},[529],[47,31794,31795,31798],{"style":17673},[47,31796],{"className":31797,"style":929},[537],[47,31799,31801],{"className":31800},[542,543,544,545],[47,31802,14413],{"className":31803,"style":3580},[508,581,545],[47,31805,31806,31809],{"style":16788},[47,31807],{"className":31808,"style":929},[537],[47,31810,31812],{"className":31811},[542,543,544,545],[47,31813,31815,31818,31821],{"className":31814},[508,545],[47,31816,27872],{"className":31817},[508,581,545],[47,31819,1163],{"className":31820},[508,581,545],[47,31822,5977],{"className":31823,"style":6420},[508,581,545],[47,31825,616],{"className":31826},[615],[47,31828,31830],{"className":31829},[525],[47,31831,31833],{"className":31832,"style":17703},[529],[47,31834],{},[47,31836,1187],{"className":31837},[512],[47,31839,31841,31844],{"className":31840},[508],[47,31842,13164],{"className":31843},[508,581],[47,31845,31847],{"className":31846},[585],[47,31848,31850,31881],{"className":31849},[520,521],[47,31851,31853,31878],{"className":31852},[525],[47,31854,31856,31867],{"className":31855,"style":17670},[529],[47,31857,31858,31861],{"style":17673},[47,31859],{"className":31860,"style":929},[537],[47,31862,31864],{"className":31863},[542,543,544,545],[47,31865,724],{"className":31866},[508,581,545],[47,31868,31869,31872],{"style":16788},[47,31870],{"className":31871,"style":929},[537],[47,31873,31875],{"className":31874},[542,543,544,545],[47,31876,1166],{"className":31877},[508,581,545],[47,31879,616],{"className":31880},[615],[47,31882,31884],{"className":31883},[525],[47,31885,31887],{"className":31886,"style":17703},[529],[47,31888],{},[47,31890,1192],{"className":31891},[1299],[47,31893],{"className":31894,"style":1303},[888],[47,31896,31898,31901],{"className":31897},[508],[47,31899,285],{"className":31900},[508,581],[47,31902,31904],{"className":31903},[585],[47,31905,31907,31938],{"className":31906},[520,521],[47,31908,31910,31935],{"className":31909},[525],[47,31911,31913,31924],{"className":31912,"style":17670},[529],[47,31914,31915,31918],{"style":17673},[47,31916],{"className":31917,"style":929},[537],[47,31919,31921],{"className":31920},[542,543,544,545],[47,31922,724],{"className":31923},[508,581,545],[47,31925,31926,31929],{"style":16788},[47,31927],{"className":31928,"style":929},[537],[47,31930,31932],{"className":31931},[542,543,544,545],[47,31933,1166],{"className":31934},[508,581,545],[47,31936,616],{"className":31937},[615],[47,31939,31941],{"className":31940},[525],[47,31942,31944],{"className":31943,"style":17703},[529],[47,31945],{},[47,31947,1201],{"className":31948},[702],[47,31950],{"className":31951,"style":1303},[888],[47,31953,14742],{"className":31954},[508],[47,31956],{"className":31957,"style":1303},[888],[47,31959,17212,31961],{"className":31960},[1338],[47,31962,17216],{"style":17215},[47,31964],{"className":31965,"style":1303},[888],[47,31967,31969,31972],{"className":31968},[508],[47,31970,5974],{"className":31971,"style":3555},[508,581],[47,31973,31975],{"className":31974},[585],[47,31976,31978,31998],{"className":31977},[520,521],[47,31979,31981,31995],{"className":31980},[525],[47,31982,31984],{"className":31983,"style":1635},[529],[47,31985,31986,31989],{"style":7830},[47,31987],{"className":31988,"style":929},[537],[47,31990,31992],{"className":31991},[542,543,544,545],[47,31993,14413],{"className":31994,"style":3580},[508,581,545],[47,31996,616],{"className":31997},[615],[47,31999,32001],{"className":32000},[525],[47,32002,32004],{"className":32003,"style":1657},[529],[47,32005],{},[47,32007,1187],{"className":32008},[512],[47,32010,32012,32015],{"className":32011},[508],[47,32013,285],{"className":32014},[508,581],[47,32016,32018],{"className":32017},[585],[47,32019,32021,32052],{"className":32020},[520,521],[47,32022,32024,32049],{"className":32023},[525],[47,32025,32027,32038],{"className":32026,"style":17670},[529],[47,32028,32029,32032],{"style":17673},[47,32030],{"className":32031,"style":929},[537],[47,32033,32035],{"className":32034},[542,543,544,545],[47,32036,724],{"className":32037},[508,581,545],[47,32039,32040,32043],{"style":16788},[47,32041],{"className":32042,"style":929},[537],[47,32044,32046],{"className":32045},[542,543,544,545],[47,32047,1166],{"className":32048},[508,581,545],[47,32050,616],{"className":32051},[615],[47,32053,32055],{"className":32054},[525],[47,32056,32058],{"className":32057,"style":17703},[529],[47,32059],{},[47,32061,13303],{"className":32062},[508],[47,32064,32066,32069],{"className":32065},[508],[47,32067,13164],{"className":32068},[508,581],[47,32070,32072],{"className":32071},[585],[47,32073,32075,32106],{"className":32074},[520,521],[47,32076,32078,32103],{"className":32077},[525],[47,32079,32081,32092],{"className":32080,"style":17670},[529],[47,32082,32083,32086],{"style":17673},[47,32084],{"className":32085,"style":929},[537],[47,32087,32089],{"className":32088},[542,543,544,545],[47,32090,724],{"className":32091},[508,581,545],[47,32093,32094,32097],{"style":16788},[47,32095],{"className":32096,"style":929},[537],[47,32098,32100],{"className":32099},[542,543,544,545],[47,32101,1166],{"className":32102},[508,581,545],[47,32104,616],{"className":32105},[615],[47,32107,32109],{"className":32108},[525],[47,32110,32112],{"className":32111,"style":17703},[529],[47,32113],{},[47,32115,1201],{"className":32116},[702],[14,32118,32119],{},"这里的价值状态函数我们一般用一个神经网路进行拟合，一般可以和策略网络公用参数，只是最后一层不同。",[10,32121,32123],{"id":32122},"proximal-policy-optimization-ppo-近邻策略优化算法","Proximal Policy Optimization (PPO) 近邻策略优化算法",[14,32125,32126],{},"在经典的强化学习环境中，我们一般是一边采集数据，一边进行训练模型，采集完的数据用完就会丢掉。这导致我们采集的数据发生了一定的浪费，且模型训练的成本也会增高。但如果我们有一种方法，能让模型在其他模型的表现中学习自己的表现，参考其他数据就能实现自身的更新，这样我们的训练成本就会大大减少。",[207,32128,32129],{"id":32129},"重要性采样",[14,32131,32132,32133,32177,32178,32206,32207,32235,32236,32264],{},"将 ",[47,32134,32136,32156],{"className":32135},[451],[47,32137,32139],{"className":32138},[455],[457,32140,32141],{"xmlns":459},[461,32142,32143,32153],{},[464,32144,32145,32147,32149,32151],{},[480,32146,12022],{},[850,32148,1187],{"stretchy":1186},[480,32150,7697],{},[850,32152,1201],{"stretchy":1186},[487,32154,32155],{"encoding":489},"f(x)",[47,32157,32159],{"className":32158,"ariaHidden":495},[494],[47,32160,32162,32165,32168,32171,32174],{"className":32161},[499],[47,32163],{"className":32164,"style":1264},[503],[47,32166,12022],{"className":32167,"style":13981},[508,581],[47,32169,1187],{"className":32170},[512],[47,32172,7697],{"className":32173},[508,581],[47,32175,1201],{"className":32176},[702]," 在分布 ",[47,32179,32181,32194],{"className":32180},[451],[47,32182,32184],{"className":32183},[455],[457,32185,32186],{"xmlns":459},[461,32187,32188,32192],{},[464,32189,32190],{},[480,32191,14],{},[487,32193,14],{"encoding":489},[47,32195,32197],{"className":32196,"ariaHidden":495},[494],[47,32198,32200,32203],{"className":32199},[499],[47,32201],{"className":32202,"style":2093},[503],[47,32204,14],{"className":32205},[508,581]," 下的期望改写为在另一分布 ",[47,32208,32210,32223],{"className":32209},[451],[47,32211,32213],{"className":32212},[455],[457,32214,32215],{"xmlns":459},[461,32216,32217,32221],{},[464,32218,32219],{},[480,32220,2081],{},[487,32222,2081],{"encoding":489},[47,32224,32226],{"className":32225,"ariaHidden":495},[494],[47,32227,32229,32232],{"className":32228},[499],[47,32230],{"className":32231,"style":2093},[503],[47,32233,2081],{"className":32234,"style":830},[508,581],"（proposal）下的期望，从而可用从 ",[47,32237,32239,32252],{"className":32238},[451],[47,32240,32242],{"className":32241},[455],[457,32243,32244],{"xmlns":459},[461,32245,32246,32250],{},[464,32247,32248],{},[480,32249,2081],{},[487,32251,2081],{"encoding":489},[47,32253,32255],{"className":32254,"ariaHidden":495},[494],[47,32256,32258,32261],{"className":32257},[499],[47,32259],{"className":32260,"style":2093},[503],[47,32262,2081],{"className":32263,"style":830},[508,581]," 采样的数据估计：",[47,32266,32268],{"className":32267},[14712],[47,32269,32271,32655],{"className":32270},[451],[47,32272,32274],{"className":32273},[455],[457,32275,32276],{"xmlns":459,"display":9848},[461,32277,32278,32652],{},[14724,32279,32280,32352,32416,32480,32552],{"rowspacing":14726,"columnalign":14727,"columnspacing":14728},[14730,32281,32282,32318],{},[14733,32283,32284],{},[14736,32285,32286],{"scriptlevel":2141,"displaystyle":495},[464,32287,32288,32290,32292,32294,32296,32298,32300],{},[480,32289,5977],{"mathvariant":858},[850,32291,1187],{"stretchy":1186},[480,32293,12022],{},[850,32295,1187],{"stretchy":1186},[480,32297,7697],{},[850,32299,1201],{"stretchy":1186},[477,32301,32302,32304],{},[850,32303,1201],{"stretchy":1186},[464,32305,32306,32308,32310,32312,32314,32316],{},[480,32307,7697],{},[850,32309,14406],{},[480,32311,14],{},[850,32313,1187],{"stretchy":1186},[480,32315,7697],{},[850,32317,1201],{"stretchy":1186},[14733,32319,32320],{},[14736,32321,32322],{"scriptlevel":2141,"displaystyle":495},[464,32323,32324,32326,32328,32334,32336,32338,32340,32342,32344,32346,32348,32350],{},[464,32325],{},[850,32327,82],{},[14423,32329,32330,32332],{},[850,32331,2213],{},[480,32333,7697],{},[480,32335,12022],{},[850,32337,1187],{"stretchy":1186},[480,32339,7697],{},[850,32341,1201],{"stretchy":1186},[850,32343,2203],{},[480,32345,14],{},[850,32347,1187],{"stretchy":1186},[480,32349,7697],{},[850,32351,1201],{"stretchy":1186},[14730,32353,32354,32360],{},[14733,32355,32356],{},[14736,32357,32358],{"scriptlevel":2141,"displaystyle":495},[464,32359],{},[14733,32361,32362],{},[14736,32363,32364],{"scriptlevel":2141,"displaystyle":495},[464,32365,32366,32368,32370,32376,32378,32380,32382,32384,32386,32388,32390,32392,32394],{},[464,32367],{},[850,32369,82],{},[14423,32371,32372,32374],{},[850,32373,2213],{},[480,32375,7697],{},[480,32377,12022],{},[850,32379,1187],{"stretchy":1186},[480,32381,7697],{},[850,32383,1201],{"stretchy":1186},[850,32385,2203],{},[480,32387,14],{},[850,32389,1187],{"stretchy":1186},[480,32391,7697],{},[850,32393,1201],{"stretchy":1186},[467,32395,32396,32406],{},[464,32397,32398,32400,32402,32404],{},[480,32399,2081],{},[850,32401,1187],{"stretchy":1186},[480,32403,7697],{},[850,32405,1201],{"stretchy":1186},[464,32407,32408,32410,32412,32414],{},[480,32409,2081],{},[850,32411,1187],{"stretchy":1186},[480,32413,7697],{},[850,32415,1201],{"stretchy":1186},[14730,32417,32418,32424],{},[14733,32419,32420],{},[14736,32421,32422],{"scriptlevel":2141,"displaystyle":495},[464,32423],{},[14733,32425,32426],{},[14736,32427,32428],{"scriptlevel":2141,"displaystyle":495},[464,32429,32430,32432,32434,32440,32442,32444,32446,32448,32470,32472,32474,32476,32478],{},[464,32431],{},[850,32433,82],{},[14423,32435,32436,32438],{},[850,32437,2213],{},[480,32439,7697],{},[480,32441,12022],{},[850,32443,1187],{"stretchy":1186},[480,32445,7697],{},[850,32447,1201],{"stretchy":1186},[467,32449,32450,32460],{},[464,32451,32452,32454,32456,32458],{},[480,32453,14],{},[850,32455,1187],{"stretchy":1186},[480,32457,7697],{},[850,32459,1201],{"stretchy":1186},[464,32461,32462,32464,32466,32468],{},[480,32463,2081],{},[850,32465,1187],{"stretchy":1186},[480,32467,7697],{},[850,32469,1201],{"stretchy":1186},[850,32471,2203],{},[480,32473,2081],{},[850,32475,1187],{"stretchy":1186},[480,32477,7697],{},[850,32479,1201],{"stretchy":1186},[14730,32481,32482,32488],{},[14733,32483,32484],{},[14736,32485,32486],{"scriptlevel":2141,"displaystyle":495},[464,32487],{},[14733,32489,32490],{},[14736,32491,32492],{"scriptlevel":2141,"displaystyle":495},[464,32493,32494,32496,32498,32500],{},[464,32495],{},[850,32497,82],{},[480,32499,5977],{"mathvariant":858},[477,32501,32502,32538],{},[464,32503,32504,32506,32508,32510,32512,32514,32536],{},[850,32505,1187],{"fence":495},[480,32507,12022],{},[850,32509,1187],{"stretchy":1186},[480,32511,7697],{},[850,32513,1201],{"stretchy":1186},[467,32515,32516,32526],{},[464,32517,32518,32520,32522,32524],{},[480,32519,14],{},[850,32521,1187],{"stretchy":1186},[480,32523,7697],{},[850,32525,1201],{"stretchy":1186},[464,32527,32528,32530,32532,32534],{},[480,32529,2081],{},[850,32531,1187],{"stretchy":1186},[480,32533,7697],{},[850,32535,1201],{"stretchy":1186},[850,32537,1201],{"fence":495},[464,32539,32540,32542,32544,32546,32548,32550],{},[480,32541,7697],{},[850,32543,14406],{},[480,32545,2081],{},[850,32547,1187],{"stretchy":1186},[480,32549,7697],{},[850,32551,1201],{"stretchy":1186},[14730,32553,32554,32560],{},[14733,32555,32556],{},[14736,32557,32558],{"scriptlevel":2141,"displaystyle":495},[464,32559],{},[14733,32561,32562],{},[14736,32563,32564],{"scriptlevel":2141,"displaystyle":495},[464,32565,32566,32568,32570,32576,32590,32592,32594,32600,32602,32632,32634,32636,32642,32644,32646,32648,32650],{},[464,32567],{},[850,32569,15037],{},[467,32571,32572,32574],{},[470,32573,472],{},[480,32575,7692],{},[15045,32577,32578,32580,32588],{},[850,32579,2213],{},[464,32581,32582,32584,32586],{},[480,32583,724],{},[850,32585,82],{},[470,32587,472],{},[480,32589,7692],{},[480,32591,12022],{},[850,32593,1187],{"stretchy":1186},[477,32595,32596,32598],{},[480,32597,7697],{},[480,32599,724],{},[850,32601,1201],{"stretchy":1186},[467,32603,32604,32618],{},[464,32605,32606,32608,32610,32616],{},[480,32607,14],{},[850,32609,1187],{"stretchy":1186},[477,32611,32612,32614],{},[480,32613,7697],{},[480,32615,724],{},[850,32617,1201],{"stretchy":1186},[464,32619,32620,32622,32624,32630],{},[480,32621,2081],{},[850,32623,1187],{"stretchy":1186},[477,32625,32626,32628],{},[480,32627,7697],{},[480,32629,724],{},[850,32631,1201],{"stretchy":1186},[850,32633,1192],{"separator":495},[888,32635],{"width":27962},[477,32637,32638,32640],{},[480,32639,7697],{},[480,32641,724],{},[850,32643,14406],{},[480,32645,2081],{},[850,32647,1187],{"stretchy":1186},[480,32649,7697],{},[850,32651,1201],{"stretchy":1186},[487,32653,32654],{"encoding":489},"\\begin{aligned}\n\\mathbb{E}(f(x))_{x \\sim p(x)} &= \\sum_{x} f(x) \\cdot p(x) \\\\\n&= \\sum_{x} f(x) \\cdot p(x) \\frac{q(x)}{q(x)} \\\\\n&= \\sum_{x} f(x) \\frac{p(x)}{q(x)} \\cdot q(x) \\\\\n&= \\mathbb{E}\\left(f(x) \\frac{p(x)}{q(x)}\\right)_{x \\sim q(x)} \\\\\n&\\approx \\frac{1}{N} \\sum_{n=1}^{N} f(x_n) \\frac{p(x_n)}{q(x_n)}, \\quad x_n \\sim q(x)\n\\end{aligned}",[47,32656,32658],{"className":32657,"ariaHidden":495},[494],[47,32659,32661,32665],{"className":32660},[499],[47,32662],{"className":32663,"style":32664},[503],"height:14.8742em;vertical-align:-7.1871em;",[47,32666,32668],{"className":32667},[508],[47,32669,32671,32818],{"className":32670},[14724],[47,32672,32674],{"className":32673},[15515],[47,32675,32677,32809],{"className":32676},[520,521],[47,32678,32680,32806],{"className":32679},[525],[47,32681,32684,32770,32779,32788,32797],{"className":32682,"style":32683},[529],"height:7.6871em;",[47,32685,32687,32691],{"style":32686},"top:-10.4654em;",[47,32688],{"className":32689,"style":32690},[537],"height:3.8283em;",[47,32692,32694,32697,32700,32703,32706,32709,32712],{"className":32693},[508],[47,32695,5977],{"className":32696},[508,910],[47,32698,1187],{"className":32699},[512],[47,32701,12022],{"className":32702,"style":13981},[508,581],[47,32704,1187],{"className":32705},[512],[47,32707,7697],{"className":32708},[508,581],[47,32710,1201],{"className":32711},[702],[47,32713,32715,32718],{"className":32714},[702],[47,32716,1201],{"className":32717},[702],[47,32719,32721],{"className":32720},[585],[47,32722,32724,32762],{"className":32723},[520,521],[47,32725,32727,32759],{"className":32726},[525],[47,32728,32730],{"className":32729,"style":595},[529],[47,32731,32732,32735],{"style":14501},[47,32733],{"className":32734,"style":929},[537],[47,32736,32738],{"className":32737},[542,543,544,545],[47,32739,32741,32744,32747,32750,32753,32756],{"className":32740},[508,545],[47,32742,7697],{"className":32743},[508,581,545],[47,32745,14406],{"className":32746},[893,545],[47,32748,14],{"className":32749},[508,581,545],[47,32751,1187],{"className":32752},[512,545],[47,32754,7697],{"className":32755},[508,581,545],[47,32757,1201],{"className":32758},[702,545],[47,32760,616],{"className":32761},[615],[47,32763,32765],{"className":32764},[525],[47,32766,32768],{"className":32767,"style":6472},[529],[47,32769],{},[47,32771,32773,32776],{"style":32772},"top:-7.4884em;",[47,32774],{"className":32775,"style":32690},[537],[47,32777],{"className":32778},[508],[47,32780,32782,32785],{"style":32781},"top:-4.5114em;",[47,32783],{"className":32784,"style":32690},[537],[47,32786],{"className":32787},[508],[47,32789,32791,32794],{"style":32790},"top:-1.5114em;",[47,32792],{"className":32793,"style":32690},[537],[47,32795],{"className":32796},[508],[47,32798,32800,32803],{"style":32799},"top:1.7917em;",[47,32801],{"className":32802,"style":32690},[537],[47,32804],{"className":32805},[508],[47,32807,616],{"className":32808},[615],[47,32810,32812],{"className":32811},[525],[47,32813,32816],{"className":32814,"style":32815},[529],"height:7.1871em;",[47,32817],{},[47,32819,32821],{"className":32820},[15739],[47,32822,32824,33921],{"className":32823},[520,521],[47,32825,32827,33918],{"className":32826},[525],[47,32828,32830,32933,33116,33299,33490],{"className":32829,"style":32683},[529],[47,32831,32832,32835],{"style":32686},[47,32833],{"className":32834,"style":32690},[537],[47,32836,32838,32841,32844,32847,32850,32897,32900,32903,32906,32909,32912,32915,32918,32921,32924,32927,32930],{"className":32837},[508],[47,32839],{"className":32840},[508],[47,32842],{"className":32843,"style":889},[888],[47,32845,82],{"className":32846},[893],[47,32848],{"className":32849,"style":889},[888],[47,32851,32853],{"className":32852},[1338,14600],[47,32854,32856,32889],{"className":32855},[520,521],[47,32857,32859,32886],{"className":32858},[525],[47,32860,32862,32876],{"className":32861,"style":15787},[529],[47,32863,32864,32867],{"style":15790},[47,32865],{"className":32866,"style":15794},[537],[47,32868,32870],{"className":32869},[542,543,544,545],[47,32871,32873],{"className":32872},[508,545],[47,32874,7697],{"className":32875},[508,581,545],[47,32877,32878,32881],{"style":15803},[47,32879],{"className":32880,"style":15794},[537],[47,32882,32883],{},[47,32884,2213],{"className":32885},[1338,2297,15812],[47,32887,616],{"className":32888},[615],[47,32890,32892],{"className":32891},[525],[47,32893,32895],{"className":32894,"style":15822},[529],[47,32896],{},[47,32898],{"className":32899,"style":1303},[888],[47,32901,12022],{"className":32902,"style":13981},[508,581],[47,32904,1187],{"className":32905},[512],[47,32907,7697],{"className":32908},[508,581],[47,32910,1201],{"className":32911},[702],[47,32913],{"className":32914,"style":2259},[888],[47,32916,2203],{"className":32917},[942],[47,32919],{"className":32920,"style":2259},[888],[47,32922,14],{"className":32923},[508,581],[47,32925,1187],{"className":32926},[512],[47,32928,7697],{"className":32929},[508,581],[47,32931,1201],{"className":32932},[702],[47,32934,32935,32938],{"style":32772},[47,32936],{"className":32937,"style":32690},[537],[47,32939,32941,32944,32947,32950,32953,33000,33003,33006,33009,33012,33015,33018,33021,33024,33027,33030,33033,33036],{"className":32940},[508],[47,32942],{"className":32943},[508],[47,32945],{"className":32946,"style":889},[888],[47,32948,82],{"className":32949},[893],[47,32951],{"className":32952,"style":889},[888],[47,32954,32956],{"className":32955},[1338,14600],[47,32957,32959,32992],{"className":32958},[520,521],[47,32960,32962,32989],{"className":32961},[525],[47,32963,32965,32979],{"className":32964,"style":15787},[529],[47,32966,32967,32970],{"style":15790},[47,32968],{"className":32969,"style":15794},[537],[47,32971,32973],{"className":32972},[542,543,544,545],[47,32974,32976],{"className":32975},[508,545],[47,32977,7697],{"className":32978},[508,581,545],[47,32980,32981,32984],{"style":15803},[47,32982],{"className":32983,"style":15794},[537],[47,32985,32986],{},[47,32987,2213],{"className":32988},[1338,2297,15812],[47,32990,616],{"className":32991},[615],[47,32993,32995],{"className":32994},[525],[47,32996,32998],{"className":32997,"style":15822},[529],[47,32999],{},[47,33001],{"className":33002,"style":1303},[888],[47,33004,12022],{"className":33005,"style":13981},[508,581],[47,33007,1187],{"className":33008},[512],[47,33010,7697],{"className":33011},[508,581],[47,33013,1201],{"className":33014},[702],[47,33016],{"className":33017,"style":2259},[888],[47,33019,2203],{"className":33020},[942],[47,33022],{"className":33023,"style":2259},[888],[47,33025,14],{"className":33026},[508,581],[47,33028,1187],{"className":33029},[512],[47,33031,7697],{"className":33032},[508,581],[47,33034,1201],{"className":33035},[702],[47,33037,33039,33042,33113],{"className":33038},[508],[47,33040],{"className":33041},[512,513],[47,33043,33045],{"className":33044},[467],[47,33046,33048,33105],{"className":33047},[520,521],[47,33049,33051,33102],{"className":33050},[525],[47,33052,33054,33074,33082],{"className":33053,"style":16178},[529],[47,33055,33056,33059],{"style":16181},[47,33057],{"className":33058,"style":538},[537],[47,33060,33062,33065,33068,33071],{"className":33061},[508],[47,33063,2081],{"className":33064,"style":830},[508,581],[47,33066,1187],{"className":33067},[512],[47,33069,7697],{"className":33070},[508,581],[47,33072,1201],{"className":33073},[702],[47,33075,33076,33079],{"style":663},[47,33077],{"className":33078,"style":538},[537],[47,33080],{"className":33081,"style":671},[670],[47,33083,33084,33087],{"style":16247},[47,33085],{"className":33086,"style":538},[537],[47,33088,33090,33093,33096,33099],{"className":33089},[508],[47,33091,2081],{"className":33092,"style":830},[508,581],[47,33094,1187],{"className":33095},[512],[47,33097,7697],{"className":33098},[508,581],[47,33100,1201],{"className":33101},[702],[47,33103,616],{"className":33104},[615],[47,33106,33108],{"className":33107},[525],[47,33109,33111],{"className":33110,"style":16312},[529],[47,33112],{},[47,33114],{"className":33115},[702,513],[47,33117,33118,33121],{"style":32781},[47,33119],{"className":33120,"style":32690},[537],[47,33122,33124,33127,33130,33133,33136,33183,33186,33189,33192,33195,33198,33278,33281,33284,33287,33290,33293,33296],{"className":33123},[508],[47,33125],{"className":33126},[508],[47,33128],{"className":33129,"style":889},[888],[47,33131,82],{"className":33132},[893],[47,33134],{"className":33135,"style":889},[888],[47,33137,33139],{"className":33138},[1338,14600],[47,33140,33142,33175],{"className":33141},[520,521],[47,33143,33145,33172],{"className":33144},[525],[47,33146,33148,33162],{"className":33147,"style":15787},[529],[47,33149,33150,33153],{"style":15790},[47,33151],{"className":33152,"style":15794},[537],[47,33154,33156],{"className":33155},[542,543,544,545],[47,33157,33159],{"className":33158},[508,545],[47,33160,7697],{"className":33161},[508,581,545],[47,33163,33164,33167],{"style":15803},[47,33165],{"className":33166,"style":15794},[537],[47,33168,33169],{},[47,33170,2213],{"className":33171},[1338,2297,15812],[47,33173,616],{"className":33174},[615],[47,33176,33178],{"className":33177},[525],[47,33179,33181],{"className":33180,"style":15822},[529],[47,33182],{},[47,33184],{"className":33185,"style":1303},[888],[47,33187,12022],{"className":33188,"style":13981},[508,581],[47,33190,1187],{"className":33191},[512],[47,33193,7697],{"className":33194},[508,581],[47,33196,1201],{"className":33197},[702],[47,33199,33201,33204,33275],{"className":33200},[508],[47,33202],{"className":33203},[512,513],[47,33205,33207],{"className":33206},[467],[47,33208,33210,33267],{"className":33209},[520,521],[47,33211,33213,33264],{"className":33212},[525],[47,33214,33216,33236,33244],{"className":33215,"style":16178},[529],[47,33217,33218,33221],{"style":16181},[47,33219],{"className":33220,"style":538},[537],[47,33222,33224,33227,33230,33233],{"className":33223},[508],[47,33225,2081],{"className":33226,"style":830},[508,581],[47,33228,1187],{"className":33229},[512],[47,33231,7697],{"className":33232},[508,581],[47,33234,1201],{"className":33235},[702],[47,33237,33238,33241],{"style":663},[47,33239],{"className":33240,"style":538},[537],[47,33242],{"className":33243,"style":671},[670],[47,33245,33246,33249],{"style":16247},[47,33247],{"className":33248,"style":538},[537],[47,33250,33252,33255,33258,33261],{"className":33251},[508],[47,33253,14],{"className":33254},[508,581],[47,33256,1187],{"className":33257},[512],[47,33259,7697],{"className":33260},[508,581],[47,33262,1201],{"className":33263},[702],[47,33265,616],{"className":33266},[615],[47,33268,33270],{"className":33269},[525],[47,33271,33273],{"className":33272,"style":16312},[529],[47,33274],{},[47,33276],{"className":33277},[702,513],[47,33279],{"className":33280,"style":2259},[888],[47,33282,2203],{"className":33283},[942],[47,33285],{"className":33286,"style":2259},[888],[47,33288,2081],{"className":33289,"style":830},[508,581],[47,33291,1187],{"className":33292},[512],[47,33294,7697],{"className":33295},[508,581],[47,33297,1201],{"className":33298},[702],[47,33300,33301,33304],{"style":32790},[47,33302],{"className":33303,"style":32690},[537],[47,33305,33307,33310,33313,33316,33319,33322,33325],{"className":33306},[508],[47,33308],{"className":33309},[508],[47,33311],{"className":33312,"style":889},[888],[47,33314,82],{"className":33315},[893],[47,33317],{"className":33318,"style":889},[888],[47,33320,5977],{"className":33321},[508,910],[47,33323],{"className":33324,"style":1303},[888],[47,33326,33328,33435],{"className":33327},[1349],[47,33329,33331,33337,33340,33343,33346,33349,33429],{"className":33330},[1349],[47,33332,33334],{"className":33333,"style":1354},[512,1353],[47,33335,1187],{"className":33336},[1358,544],[47,33338,12022],{"className":33339,"style":13981},[508,581],[47,33341,1187],{"className":33342},[512],[47,33344,7697],{"className":33345},[508,581],[47,33347,1201],{"className":33348},[702],[47,33350,33352,33355,33426],{"className":33351},[508],[47,33353],{"className":33354},[512,513],[47,33356,33358],{"className":33357},[467],[47,33359,33361,33418],{"className":33360},[520,521],[47,33362,33364,33415],{"className":33363},[525],[47,33365,33367,33387,33395],{"className":33366,"style":16178},[529],[47,33368,33369,33372],{"style":16181},[47,33370],{"className":33371,"style":538},[537],[47,33373,33375,33378,33381,33384],{"className":33374},[508],[47,33376,2081],{"className":33377,"style":830},[508,581],[47,33379,1187],{"className":33380},[512],[47,33382,7697],{"className":33383},[508,581],[47,33385,1201],{"className":33386},[702],[47,33388,33389,33392],{"style":663},[47,33390],{"className":33391,"style":538},[537],[47,33393],{"className":33394,"style":671},[670],[47,33396,33397,33400],{"style":16247},[47,33398],{"className":33399,"style":538},[537],[47,33401,33403,33406,33409,33412],{"className":33402},[508],[47,33404,14],{"className":33405},[508,581],[47,33407,1187],{"className":33408},[512],[47,33410,7697],{"className":33411},[508,581],[47,33413,1201],{"className":33414},[702],[47,33416,616],{"className":33417},[615],[47,33419,33421],{"className":33420},[525],[47,33422,33424],{"className":33423,"style":16312},[529],[47,33425],{},[47,33427],{"className":33428},[702,513],[47,33430,33432],{"className":33431,"style":1354},[702,1353],[47,33433,1201],{"className":33434},[1358,544],[47,33436,33438],{"className":33437},[585],[47,33439,33441,33481],{"className":33440},[520,521],[47,33442,33444,33478],{"className":33443},[525],[47,33445,33448],{"className":33446,"style":33447},[529],"height:-0.4747em;",[47,33449,33451,33454],{"style":33450},"top:-1.7003em;margin-right:0.05em;",[47,33452],{"className":33453,"style":929},[537],[47,33455,33457],{"className":33456},[542,543,544,545],[47,33458,33460,33463,33466,33469,33472,33475],{"className":33459},[508,545],[47,33461,7697],{"className":33462},[508,581,545],[47,33464,14406],{"className":33465},[893,545],[47,33467,2081],{"className":33468,"style":830},[508,581,545],[47,33470,1187],{"className":33471},[512,545],[47,33473,7697],{"className":33474},[508,581,545],[47,33476,1201],{"className":33477},[702,545],[47,33479,616],{"className":33480},[615],[47,33482,33484],{"className":33483},[525],[47,33485,33488],{"className":33486,"style":33487},[529],"height:1.1747em;",[47,33489],{},[47,33491,33492,33495],{"style":32799},[47,33493],{"className":33494,"style":32690},[537],[47,33496,33498,33501,33504,33507,33510,33572,33575,33642,33645,33648,33651,33691,33694,33848,33851,33854,33857,33897,33900,33903,33906,33909,33912,33915],{"className":33497},[508],[47,33499],{"className":33500},[508],[47,33502],{"className":33503,"style":889},[888],[47,33505,15037],{"className":33506},[893],[47,33508],{"className":33509,"style":889},[888],[47,33511,33513,33516,33569],{"className":33512},[508],[47,33514],{"className":33515},[512,513],[47,33517,33519],{"className":33518},[467],[47,33520,33522,33561],{"className":33521},[520,521],[47,33523,33525,33558],{"className":33524},[525],[47,33526,33528,33539,33547],{"className":33527,"style":16641},[529],[47,33529,33530,33533],{"style":16181},[47,33531],{"className":33532,"style":538},[537],[47,33534,33536],{"className":33535},[508],[47,33537,7692],{"className":33538,"style":7767},[508,581],[47,33540,33541,33544],{"style":663},[47,33542],{"className":33543,"style":538},[537],[47,33545],{"className":33546,"style":671},[670],[47,33548,33549,33552],{"style":16247},[47,33550],{"className":33551,"style":538},[537],[47,33553,33555],{"className":33554},[508],[47,33556,472],{"className":33557},[508],[47,33559,616],{"className":33560},[615],[47,33562,33564],{"className":33563},[525],[47,33565,33567],{"className":33566,"style":16681},[529],[47,33568],{},[47,33570],{"className":33571},[702,513],[47,33573],{"className":33574,"style":1303},[888],[47,33576,33578],{"className":33577},[1338,14600],[47,33579,33581,33634],{"className":33580},[520,521],[47,33582,33584,33631],{"className":33583},[525],[47,33585,33587,33607,33617],{"className":33586,"style":16702},[529],[47,33588,33589,33592],{"style":16705},[47,33590],{"className":33591,"style":15794},[537],[47,33593,33595],{"className":33594},[542,543,544,545],[47,33596,33598,33601,33604],{"className":33597},[508,545],[47,33599,724],{"className":33600},[508,581,545],[47,33602,82],{"className":33603},[893,545],[47,33605,472],{"className":33606},[508,545],[47,33608,33609,33612],{"style":15803},[47,33610],{"className":33611,"style":15794},[537],[47,33613,33614],{},[47,33615,2213],{"className":33616},[1338,2297,15812],[47,33618,33619,33622],{"style":16736},[47,33620],{"className":33621,"style":15794},[537],[47,33623,33625],{"className":33624},[542,543,544,545],[47,33626,33628],{"className":33627},[508,545],[47,33629,7692],{"className":33630,"style":7767},[508,581,545],[47,33632,616],{"className":33633},[615],[47,33635,33637],{"className":33636},[525],[47,33638,33640],{"className":33639,"style":16755},[529],[47,33641],{},[47,33643],{"className":33644,"style":1303},[888],[47,33646,12022],{"className":33647,"style":13981},[508,581],[47,33649,1187],{"className":33650},[512],[47,33652,33654,33657],{"className":33653},[508],[47,33655,7697],{"className":33656},[508,581],[47,33658,33660],{"className":33659},[585],[47,33661,33663,33683],{"className":33662},[520,521],[47,33664,33666,33680],{"className":33665},[525],[47,33667,33669],{"className":33668,"style":1709},[529],[47,33670,33671,33674],{"style":1638},[47,33672],{"className":33673,"style":929},[537],[47,33675,33677],{"className":33676},[542,543,544,545],[47,33678,724],{"className":33679},[508,581,545],[47,33681,616],{"className":33682},[615],[47,33684,33686],{"className":33685},[525],[47,33687,33689],{"className":33688,"style":1657},[529],[47,33690],{},[47,33692,1201],{"className":33693},[702],[47,33695,33697,33700,33845],{"className":33696},[508],[47,33698],{"className":33699},[512,513],[47,33701,33703],{"className":33702},[467],[47,33704,33706,33837],{"className":33705},[520,521],[47,33707,33709,33834],{"className":33708},[525],[47,33710,33712,33769,33777],{"className":33711,"style":16178},[529],[47,33713,33714,33717],{"style":16181},[47,33715],{"className":33716,"style":538},[537],[47,33718,33720,33723,33726,33766],{"className":33719},[508],[47,33721,2081],{"className":33722,"style":830},[508,581],[47,33724,1187],{"className":33725},[512],[47,33727,33729,33732],{"className":33728},[508],[47,33730,7697],{"className":33731},[508,581],[47,33733,33735],{"className":33734},[585],[47,33736,33738,33758],{"className":33737},[520,521],[47,33739,33741,33755],{"className":33740},[525],[47,33742,33744],{"className":33743,"style":1709},[529],[47,33745,33746,33749],{"style":1638},[47,33747],{"className":33748,"style":929},[537],[47,33750,33752],{"className":33751},[542,543,544,545],[47,33753,724],{"className":33754},[508,581,545],[47,33756,616],{"className":33757},[615],[47,33759,33761],{"className":33760},[525],[47,33762,33764],{"className":33763,"style":1657},[529],[47,33765],{},[47,33767,1201],{"className":33768},[702],[47,33770,33771,33774],{"style":663},[47,33772],{"className":33773,"style":538},[537],[47,33775],{"className":33776,"style":671},[670],[47,33778,33779,33782],{"style":16247},[47,33780],{"className":33781,"style":538},[537],[47,33783,33785,33788,33791,33831],{"className":33784},[508],[47,33786,14],{"className":33787},[508,581],[47,33789,1187],{"className":33790},[512],[47,33792,33794,33797],{"className":33793},[508],[47,33795,7697],{"className":33796},[508,581],[47,33798,33800],{"className":33799},[585],[47,33801,33803,33823],{"className":33802},[520,521],[47,33804,33806,33820],{"className":33805},[525],[47,33807,33809],{"className":33808,"style":1709},[529],[47,33810,33811,33814],{"style":1638},[47,33812],{"className":33813,"style":929},[537],[47,33815,33817],{"className":33816},[542,543,544,545],[47,33818,724],{"className":33819},[508,581,545],[47,33821,616],{"className":33822},[615],[47,33824,33826],{"className":33825},[525],[47,33827,33829],{"className":33828,"style":1657},[529],[47,33830],{},[47,33832,1201],{"className":33833},[702],[47,33835,616],{"className":33836},[615],[47,33838,33840],{"className":33839},[525],[47,33841,33843],{"className":33842,"style":16312},[529],[47,33844],{},[47,33846],{"className":33847},[702,513],[47,33849,1192],{"className":33850},[1299],[47,33852],{"className":33853,"style":28927},[888],[47,33855],{"className":33856,"style":1303},[888],[47,33858,33860,33863],{"className":33859},[508],[47,33861,7697],{"className":33862},[508,581],[47,33864,33866],{"className":33865},[585],[47,33867,33869,33889],{"className":33868},[520,521],[47,33870,33872,33886],{"className":33871},[525],[47,33873,33875],{"className":33874,"style":1709},[529],[47,33876,33877,33880],{"style":1638},[47,33878],{"className":33879,"style":929},[537],[47,33881,33883],{"className":33882},[542,543,544,545],[47,33884,724],{"className":33885},[508,581,545],[47,33887,616],{"className":33888},[615],[47,33890,33892],{"className":33891},[525],[47,33893,33895],{"className":33894,"style":1657},[529],[47,33896],{},[47,33898],{"className":33899,"style":889},[888],[47,33901,14406],{"className":33902},[893],[47,33904],{"className":33905,"style":889},[888],[47,33907,2081],{"className":33908,"style":830},[508,581],[47,33910,1187],{"className":33911},[512],[47,33913,7697],{"className":33914},[508,581],[47,33916,1201],{"className":33917},[702],[47,33919,616],{"className":33920},[615],[47,33922,33924],{"className":33923},[525],[47,33925,33927],{"className":33926,"style":32815},[529],[47,33928],{},[14,33930,33931],{},"这样我们可以用重要性采样更新我们的目标函数的梯度公式，这样我们可以把 on-policy 的训练改为 off-policy 的训练。",[207,33933,33935],{"id":33934},"off-policy","Off-policy",[14,33937,33938,33939,34002,34003,34032,34033,34170,34171,34362,34363,34578],{},"记 ",[47,33940,33942,33960],{"className":33941},[451],[47,33943,33945],{"className":33944},[455],[457,33946,33947],{"xmlns":459},[461,33948,33949,33957],{},[464,33950,33951],{},[854,33952,33953,33955],{},[480,33954,14413],{},[850,33956,19346],{"mathvariant":1206,"lspace":14728,"rspace":14728},[487,33958,33959],{"encoding":489},"\\theta'",[47,33961,33963],{"className":33962,"ariaHidden":495},[494],[47,33964,33966,33970],{"className":33965},[499],[47,33967],{"className":33968,"style":33969},[503],"height:0.7519em;",[47,33971,33973,33976],{"className":33972},[508],[47,33974,14413],{"className":33975,"style":3580},[508,581],[47,33977,33979],{"className":33978},[585],[47,33980,33982],{"className":33981},[520],[47,33983,33985],{"className":33984},[525],[47,33986,33988],{"className":33987,"style":33969},[529],[47,33989,33990,33993],{"style":925},[47,33991],{"className":33992,"style":929},[537],[47,33994,33996],{"className":33995},[542,543,544,545],[47,33997,33999],{"className":33998},[508,545],[47,34000,19346],{"className":34001},[508,545]," 为采集数据时的旧策略，",[47,34004,34006,34020],{"className":34005},[451],[47,34007,34009],{"className":34008},[455],[457,34010,34011],{"xmlns":459},[461,34012,34013,34017],{},[464,34014,34015],{},[480,34016,14413],{},[487,34018,34019],{"encoding":489},"\\theta",[47,34021,34023],{"className":34022,"ariaHidden":495},[494],[47,34024,34026,34029],{"className":34025},[499],[47,34027],{"className":34028,"style":796},[503],[47,34030,14413],{"className":34031,"style":3580},[508,581]," 为当前要优化的策略；优势 ",[47,34034,34036,34066],{"className":34035},[451],[47,34037,34039],{"className":34038},[455],[457,34040,34041],{"xmlns":459},[461,34042,34043,34063],{},[464,34044,34045],{},[2209,34046,34047,34049,34055],{},[480,34048,1163],{},[854,34050,34051,34053],{},[480,34052,14413],{},[850,34054,19346],{"mathvariant":1206,"lspace":14728,"rspace":14728},[464,34056,34057,34059,34061],{},[480,34058,27872],{},[480,34060,1163],{},[480,34062,5977],{},[487,34064,34065],{"encoding":489},"A_{\\theta'}^{GAE}",[47,34067,34069],{"className":34068,"ariaHidden":495},[494],[47,34070,34072,34076],{"className":34071},[499],[47,34073],{"className":34074,"style":34075},[503],"height:1.1244em;vertical-align:-0.2831em;",[47,34077,34079,34082],{"className":34078},[508],[47,34080,1163],{"className":34081},[508,581],[47,34083,34085],{"className":34084},[585],[47,34086,34088,34161],{"className":34087},[520,521],[47,34089,34091,34158],{"className":34090},[525],[47,34092,34094,34138],{"className":34093,"style":3568},[529],[47,34095,34097,34100],{"style":34096},"top:-2.4169em;margin-left:0em;margin-right:0.05em;",[47,34098],{"className":34099,"style":929},[537],[47,34101,34103],{"className":34102},[542,543,544,545],[47,34104,34106],{"className":34105},[508,545],[47,34107,34109,34112],{"className":34108},[508,545],[47,34110,14413],{"className":34111,"style":3580},[508,581,545],[47,34113,34115],{"className":34114},[585],[47,34116,34118],{"className":34117},[520],[47,34119,34121],{"className":34120},[525],[47,34122,34124],{"className":34123,"style":19506},[529],[47,34125,34126,34129],{"style":19509},[47,34127],{"className":34128,"style":602},[537],[47,34130,34132],{"className":34131},[542,606,607,545],[47,34133,34135],{"className":34134},[508,545],[47,34136,19346],{"className":34137},[508,545],[47,34139,34140,34143],{"style":925},[47,34141],{"className":34142,"style":929},[537],[47,34144,34146],{"className":34145},[542,543,544,545],[47,34147,34149,34152,34155],{"className":34148},[508,545],[47,34150,27872],{"className":34151},[508,581,545],[47,34153,1163],{"className":34154},[508,581,545],[47,34156,5977],{"className":34157,"style":6420},[508,581,545],[47,34159,616],{"className":34160},[615],[47,34162,34164],{"className":34163},[525],[47,34165,34168],{"className":34166,"style":34167},[529],"height:0.2831em;",[47,34169],{}," 由旧策略下的价值网络估计。利用恒等式 ",[47,34172,34174,34228],{"className":34173},[451],[47,34175,34177],{"className":34176},[455],[457,34178,34179],{"xmlns":459},[461,34180,34181,34225],{},[464,34182,34183,34185,34187,34189,34191,34193,34195,34197,34199],{},[480,34184,14742],{"mathvariant":1206},[480,34186,15167],{},[850,34188,1210],{},[480,34190,12022],{},[850,34192,1187],{"stretchy":1186},[480,34194,7697],{},[850,34196,1201],{"stretchy":1186},[850,34198,82],{},[14736,34200,34201],{"scriptlevel":2141,"displaystyle":495},[467,34202,34203,34215],{},[464,34204,34205,34207,34209,34211,34213],{},[480,34206,14742],{"mathvariant":1206},[480,34208,12022],{},[850,34210,1187],{"stretchy":1186},[480,34212,7697],{},[850,34214,1201],{"stretchy":1186},[464,34216,34217,34219,34221,34223],{},[480,34218,12022],{},[850,34220,1187],{"stretchy":1186},[480,34222,7697],{},[850,34224,1201],{"stretchy":1186},[487,34226,34227],{"encoding":489},"\\nabla \\log f(x) = \\dfrac{\\nabla f(x)}{f(x)}",[47,34229,34231,34272],{"className":34230,"ariaHidden":495},[494],[47,34232,34234,34237,34240,34243,34248,34251,34254,34257,34260,34263,34266,34269],{"className":34233},[499],[47,34235],{"className":34236,"style":1264},[503],[47,34238,14742],{"className":34239},[508],[47,34241],{"className":34242,"style":1303},[888],[47,34244,17212,34246],{"className":34245},[1338],[47,34247,17216],{"style":17215},[47,34249],{"className":34250,"style":1303},[888],[47,34252,12022],{"className":34253,"style":13981},[508,581],[47,34255,1187],{"className":34256},[512],[47,34258,7697],{"className":34259},[508,581],[47,34261,1201],{"className":34262},[702],[47,34264],{"className":34265,"style":889},[888],[47,34267,82],{"className":34268},[893],[47,34270],{"className":34271,"style":889},[888],[47,34273,34275,34279],{"className":34274},[499],[47,34276],{"className":34277,"style":34278},[503],"height:2.363em;vertical-align:-0.936em;",[47,34280,34282,34285,34359],{"className":34281},[508],[47,34283],{"className":34284},[512,513],[47,34286,34288],{"className":34287},[467],[47,34289,34291,34351],{"className":34290},[520,521],[47,34292,34294,34348],{"className":34293},[525],[47,34295,34297,34317,34325],{"className":34296,"style":16178},[529],[47,34298,34299,34302],{"style":16181},[47,34300],{"className":34301,"style":538},[537],[47,34303,34305,34308,34311,34314],{"className":34304},[508],[47,34306,12022],{"className":34307,"style":13981},[508,581],[47,34309,1187],{"className":34310},[512],[47,34312,7697],{"className":34313},[508,581],[47,34315,1201],{"className":34316},[702],[47,34318,34319,34322],{"style":663},[47,34320],{"className":34321,"style":538},[537],[47,34323],{"className":34324,"style":671},[670],[47,34326,34327,34330],{"style":16247},[47,34328],{"className":34329,"style":538},[537],[47,34331,34333,34336,34339,34342,34345],{"className":34332},[508],[47,34334,14742],{"className":34335},[508],[47,34337,12022],{"className":34338,"style":13981},[508,581],[47,34340,1187],{"className":34341},[512],[47,34343,7697],{"className":34344},[508,581],[47,34346,1201],{"className":34347},[702],[47,34349,616],{"className":34350},[615],[47,34352,34354],{"className":34353},[525],[47,34355,34357],{"className":34356,"style":16312},[529],[47,34358],{},[47,34360],{"className":34361},[702,513],"，可将策略梯度写成重要性采样比 ",[47,34364,34366,34398],{"className":34365},[451],[47,34367,34369],{"className":34368},[455],[457,34370,34371],{"xmlns":459},[461,34372,34373,34395],{},[464,34374,34375],{},[14736,34376,34377],{"scriptlevel":2141,"displaystyle":495},[467,34378,34379,34385],{},[477,34380,34381,34383],{},[480,34382,5974],{},[480,34384,14413],{},[477,34386,34387,34389],{},[480,34388,5974],{},[854,34390,34391,34393],{},[480,34392,14413],{},[850,34394,19346],{"mathvariant":1206,"lspace":14728,"rspace":14728},[487,34396,34397],{"encoding":489},"\\dfrac{P_\\theta}{P_{\\theta'}}",[47,34399,34401],{"className":34400,"ariaHidden":495},[494],[47,34402,34404,34408],{"className":34403},[499],[47,34405],{"className":34406,"style":34407},[503],"height:2.1963em;vertical-align:-0.836em;",[47,34409,34411,34414,34575],{"className":34410},[508],[47,34412],{"className":34413},[512,513],[47,34415,34417],{"className":34416},[467],[47,34418,34420,34566],{"className":34419},[520,521],[47,34421,34423,34563],{"className":34422},[525],[47,34424,34427,34507,34515],{"className":34425,"style":34426},[529],"height:1.3603em;",[47,34428,34429,34432],{"style":16181},[47,34430],{"className":34431,"style":538},[537],[47,34433,34435],{"className":34434},[508],[47,34436,34438,34441],{"className":34437},[508],[47,34439,5974],{"className":34440,"style":3555},[508,581],[47,34442,34444],{"className":34443},[585],[47,34445,34447,34499],{"className":34446},[520,521],[47,34448,34450,34496],{"className":34449},[525],[47,34451,34453],{"className":34452,"style":1635},[529],[47,34454,34455,34458],{"style":7830},[47,34456],{"className":34457,"style":929},[537],[47,34459,34461],{"className":34460},[542,543,544,545],[47,34462,34464],{"className":34463},[508,545],[47,34465,34467,34470],{"className":34466},[508,545],[47,34468,14413],{"className":34469,"style":3580},[508,581,545],[47,34471,34473],{"className":34472},[585],[47,34474,34476],{"className":34475},[520],[47,34477,34479],{"className":34478},[525],[47,34480,34482],{"className":34481,"style":19506},[529],[47,34483,34484,34487],{"style":19509},[47,34485],{"className":34486,"style":602},[537],[47,34488,34490],{"className":34489},[542,606,607,545],[47,34491,34493],{"className":34492},[508,545],[47,34494,19346],{"className":34495},[508,545],[47,34497,616],{"className":34498},[615],[47,34500,34502],{"className":34501},[525],[47,34503,34505],{"className":34504,"style":1657},[529],[47,34506],{},[47,34508,34509,34512],{"style":663},[47,34510],{"className":34511,"style":538},[537],[47,34513],{"className":34514,"style":671},[670],[47,34516,34517,34520],{"style":16247},[47,34518],{"className":34519,"style":538},[537],[47,34521,34523],{"className":34522},[508],[47,34524,34526,34529],{"className":34525},[508],[47,34527,5974],{"className":34528,"style":3555},[508,581],[47,34530,34532],{"className":34531},[585],[47,34533,34535,34555],{"className":34534},[520,521],[47,34536,34538,34552],{"className":34537},[525],[47,34539,34541],{"className":34540,"style":1635},[529],[47,34542,34543,34546],{"style":7830},[47,34544],{"className":34545,"style":929},[537],[47,34547,34549],{"className":34548},[542,543,544,545],[47,34550,14413],{"className":34551,"style":3580},[508,581,545],[47,34553,616],{"className":34554},[615],[47,34556,34558],{"className":34557},[525],[47,34559,34561],{"className":34560,"style":1657},[529],[47,34562],{},[47,34564,616],{"className":34565},[615],[47,34567,34569],{"className":34568},[525],[47,34570,34573],{"className":34571,"style":34572},[529],"height:0.836em;",[47,34574],{},[47,34576],{"className":34577},[702,513]," 的形式：",[47,34580,34582],{"className":34581},[14712],[47,34583,34585,34637],{"className":34584},[451],[47,34586,34588],{"className":34587},[455],[457,34589,34590],{"xmlns":459,"display":9848},[461,34591,34592,34634],{},[464,34593,34594,34596,34598,34600,34602,34604,34606,34608,34610],{},[480,34595,14742],{"mathvariant":1206},[480,34597,15167],{},[850,34599,1210],{},[480,34601,12022],{},[850,34603,1187],{"stretchy":1186},[480,34605,7697],{},[850,34607,1201],{"stretchy":1186},[850,34609,82],{},[467,34611,34612,34624],{},[464,34613,34614,34616,34618,34620,34622],{},[480,34615,14742],{"mathvariant":1206},[480,34617,12022],{},[850,34619,1187],{"stretchy":1186},[480,34621,7697],{},[850,34623,1201],{"stretchy":1186},[464,34625,34626,34628,34630,34632],{},[480,34627,12022],{},[850,34629,1187],{"stretchy":1186},[480,34631,7697],{},[850,34633,1201],{"stretchy":1186},[487,34635,34636],{"encoding":489},"\\nabla \\log f(x) = \\frac{\\nabla f(x)}{f(x)}",[47,34638,34640,34681],{"className":34639,"ariaHidden":495},[494],[47,34641,34643,34646,34649,34652,34657,34660,34663,34666,34669,34672,34675,34678],{"className":34642},[499],[47,34644],{"className":34645,"style":1264},[503],[47,34647,14742],{"className":34648},[508],[47,34650],{"className":34651,"style":1303},[888],[47,34653,17212,34655],{"className":34654},[1338],[47,34656,17216],{"style":17215},[47,34658],{"className":34659,"style":1303},[888],[47,34661,12022],{"className":34662,"style":13981},[508,581],[47,34664,1187],{"className":34665},[512],[47,34667,7697],{"className":34668},[508,581],[47,34670,1201],{"className":34671},[702],[47,34673],{"className":34674,"style":889},[888],[47,34676,82],{"className":34677},[893],[47,34679],{"className":34680,"style":889},[888],[47,34682,34684,34687],{"className":34683},[499],[47,34685],{"className":34686,"style":34278},[503],[47,34688,34690,34693,34767],{"className":34689},[508],[47,34691],{"className":34692},[512,513],[47,34694,34696],{"className":34695},[467],[47,34697,34699,34759],{"className":34698},[520,521],[47,34700,34702,34756],{"className":34701},[525],[47,34703,34705,34725,34733],{"className":34704,"style":16178},[529],[47,34706,34707,34710],{"style":16181},[47,34708],{"className":34709,"style":538},[537],[47,34711,34713,34716,34719,34722],{"className":34712},[508],[47,34714,12022],{"className":34715,"style":13981},[508,581],[47,34717,1187],{"className":34718},[512],[47,34720,7697],{"className":34721},[508,581],[47,34723,1201],{"className":34724},[702],[47,34726,34727,34730],{"style":663},[47,34728],{"className":34729,"style":538},[537],[47,34731],{"className":34732,"style":671},[670],[47,34734,34735,34738],{"style":16247},[47,34736],{"className":34737,"style":538},[537],[47,34739,34741,34744,34747,34750,34753],{"className":34740},[508],[47,34742,14742],{"className":34743},[508],[47,34745,12022],{"className":34746,"style":13981},[508,581],[47,34748,1187],{"className":34749},[512],[47,34751,7697],{"className":34752},[508,581],[47,34754,1201],{"className":34755},[702],[47,34757,616],{"className":34758},[615],[47,34760,34762],{"className":34761},[525],[47,34763,34765],{"className":34764,"style":16312},[529],[47,34766],{},[47,34768],{"className":34769},[702,513],[47,34771,34773],{"className":34772},[14712],[47,34774,34776,35498],{"className":34775},[451],[47,34777,34779],{"className":34778},[455],[457,34780,34781],{"xmlns":459,"display":9848},[461,34782,34783,35495],{},[14724,34784,34785,34909,35105,35331],{"rowspacing":14726,"columnalign":14727,"columnspacing":14728},[14730,34786,34787,34793],{},[14733,34788,34789],{},[14736,34790,34791],{"scriptlevel":2141,"displaystyle":495},[464,34792],{},[14733,34794,34795],{},[14736,34796,34797],{"scriptlevel":2141,"displaystyle":495},[464,34798,34799,34801,34807,34821,34839,34853,34855,34863,34865,34873,34875,34877,34879,34881,34887,34889,34897,34899,34907],{},[464,34800],{},[467,34802,34803,34805],{},[470,34804,472],{},[480,34806,7692],{},[15045,34808,34809,34811,34819],{},[850,34810,2213],{},[464,34812,34813,34815,34817],{},[480,34814,724],{},[850,34816,82],{},[470,34818,472],{},[480,34820,7692],{},[15045,34822,34823,34825,34833],{},[850,34824,2213],{},[464,34826,34827,34829,34831],{},[480,34828,1166],{},[850,34830,82],{},[470,34832,472],{},[477,34834,34835,34837],{},[480,34836,13604],{},[480,34838,724],{},[2209,34840,34841,34843,34845],{},[480,34842,1163],{},[480,34844,14413],{},[464,34846,34847,34849,34851],{},[480,34848,27872],{},[480,34850,1163],{},[480,34852,5977],{},[850,34854,1187],{"stretchy":1186},[2209,34856,34857,34859,34861],{},[480,34858,13164],{},[480,34860,724],{},[480,34862,1166],{},[850,34864,1192],{"separator":495},[2209,34866,34867,34869,34871],{},[480,34868,285],{},[480,34870,724],{},[480,34872,1166],{},[850,34874,1201],{"stretchy":1186},[480,34876,14742],{"mathvariant":1206},[480,34878,15167],{},[850,34880,1210],{},[477,34882,34883,34885],{},[480,34884,5974],{},[480,34886,14413],{},[850,34888,1187],{"stretchy":1186},[2209,34890,34891,34893,34895],{},[480,34892,285],{},[480,34894,724],{},[480,34896,1166],{},[480,34898,13303],{"mathvariant":1206},[2209,34900,34901,34903,34905],{},[480,34902,13164],{},[480,34904,724],{},[480,34906,1166],{},[850,34908,1201],{"stretchy":1186},[14730,34910,34911,34917],{},[14733,34912,34913],{},[14736,34914,34915],{"scriptlevel":2141,"displaystyle":495},[464,34916],{},[14733,34918,34919],{},[14736,34920,34921],{"scriptlevel":2141,"displaystyle":495},[464,34922,34923,34925,34927,34933,34947,34965,34983,34985,34993,34995,35003,35005,35071,35073,35075,35077,35083,35085,35093,35095,35103],{},[464,34924],{},[850,34926,82],{},[467,34928,34929,34931],{},[470,34930,472],{},[480,34932,7692],{},[15045,34934,34935,34937,34945],{},[850,34936,2213],{},[464,34938,34939,34941,34943],{},[480,34940,724],{},[850,34942,82],{},[470,34944,472],{},[480,34946,7692],{},[15045,34948,34949,34951,34959],{},[850,34950,2213],{},[464,34952,34953,34955,34957],{},[480,34954,1166],{},[850,34956,82],{},[470,34958,472],{},[477,34960,34961,34963],{},[480,34962,13604],{},[480,34964,724],{},[2209,34966,34967,34969,34975],{},[480,34968,1163],{},[854,34970,34971,34973],{},[480,34972,14413],{},[850,34974,19346],{"mathvariant":1206,"lspace":14728,"rspace":14728},[464,34976,34977,34979,34981],{},[480,34978,27872],{},[480,34980,1163],{},[480,34982,5977],{},[850,34984,1187],{"stretchy":1186},[2209,34986,34987,34989,34991],{},[480,34988,13164],{},[480,34990,724],{},[480,34992,1166],{},[850,34994,1192],{"separator":495},[2209,34996,34997,34999,35001],{},[480,34998,285],{},[480,35000,724],{},[480,35002,1166],{},[850,35004,1201],{"stretchy":1186},[467,35006,35007,35037],{},[464,35008,35009,35015,35017,35025,35027,35035],{},[477,35010,35011,35013],{},[480,35012,5974],{},[480,35014,14413],{},[850,35016,1187],{"stretchy":1186},[2209,35018,35019,35021,35023],{},[480,35020,285],{},[480,35022,724],{},[480,35024,1166],{},[480,35026,13303],{"mathvariant":1206},[2209,35028,35029,35031,35033],{},[480,35030,13164],{},[480,35032,724],{},[480,35034,1166],{},[850,35036,1201],{"stretchy":1186},[464,35038,35039,35049,35051,35059,35061,35069],{},[477,35040,35041,35043],{},[480,35042,5974],{},[854,35044,35045,35047],{},[480,35046,14413],{},[850,35048,19346],{"mathvariant":1206,"lspace":14728,"rspace":14728},[850,35050,1187],{"stretchy":1186},[2209,35052,35053,35055,35057],{},[480,35054,285],{},[480,35056,724],{},[480,35058,1166],{},[480,35060,13303],{"mathvariant":1206},[2209,35062,35063,35065,35067],{},[480,35064,13164],{},[480,35066,724],{},[480,35068,1166],{},[850,35070,1201],{"stretchy":1186},[480,35072,14742],{"mathvariant":1206},[480,35074,15167],{},[850,35076,1210],{},[477,35078,35079,35081],{},[480,35080,5974],{},[480,35082,14413],{},[850,35084,1187],{"stretchy":1186},[2209,35086,35087,35089,35091],{},[480,35088,285],{},[480,35090,724],{},[480,35092,1166],{},[480,35094,13303],{"mathvariant":1206},[2209,35096,35097,35099,35101],{},[480,35098,13164],{},[480,35100,724],{},[480,35102,1166],{},[850,35104,1201],{"stretchy":1186},[14730,35106,35107,35113],{},[14733,35108,35109],{},[14736,35110,35111],{"scriptlevel":2141,"displaystyle":495},[464,35112],{},[14733,35114,35115],{},[14736,35116,35117],{"scriptlevel":2141,"displaystyle":495},[464,35118,35119,35121,35123,35129,35143,35161,35179,35181,35189,35191,35199,35201,35267],{},[464,35120],{},[850,35122,82],{},[467,35124,35125,35127],{},[470,35126,472],{},[480,35128,7692],{},[15045,35130,35131,35133,35141],{},[850,35132,2213],{},[464,35134,35135,35137,35139],{},[480,35136,724],{},[850,35138,82],{},[470,35140,472],{},[480,35142,7692],{},[15045,35144,35145,35147,35155],{},[850,35146,2213],{},[464,35148,35149,35151,35153],{},[480,35150,1166],{},[850,35152,82],{},[470,35154,472],{},[477,35156,35157,35159],{},[480,35158,13604],{},[480,35160,724],{},[2209,35162,35163,35165,35171],{},[480,35164,1163],{},[854,35166,35167,35169],{},[480,35168,14413],{},[850,35170,19346],{"mathvariant":1206,"lspace":14728,"rspace":14728},[464,35172,35173,35175,35177],{},[480,35174,27872],{},[480,35176,1163],{},[480,35178,5977],{},[850,35180,1187],{"stretchy":1186},[2209,35182,35183,35185,35187],{},[480,35184,13164],{},[480,35186,724],{},[480,35188,1166],{},[850,35190,1192],{"separator":495},[2209,35192,35193,35195,35197],{},[480,35194,285],{},[480,35196,724],{},[480,35198,1166],{},[850,35200,1201],{"stretchy":1186},[467,35202,35203,35233],{},[464,35204,35205,35211,35213,35221,35223,35231],{},[477,35206,35207,35209],{},[480,35208,5974],{},[480,35210,14413],{},[850,35212,1187],{"stretchy":1186},[2209,35214,35215,35217,35219],{},[480,35216,285],{},[480,35218,724],{},[480,35220,1166],{},[480,35222,13303],{"mathvariant":1206},[2209,35224,35225,35227,35229],{},[480,35226,13164],{},[480,35228,724],{},[480,35230,1166],{},[850,35232,1201],{"stretchy":1186},[464,35234,35235,35245,35247,35255,35257,35265],{},[477,35236,35237,35239],{},[480,35238,5974],{},[854,35240,35241,35243],{},[480,35242,14413],{},[850,35244,19346],{"mathvariant":1206,"lspace":14728,"rspace":14728},[850,35246,1187],{"stretchy":1186},[2209,35248,35249,35251,35253],{},[480,35250,285],{},[480,35252,724],{},[480,35254,1166],{},[480,35256,13303],{"mathvariant":1206},[2209,35258,35259,35261,35263],{},[480,35260,13164],{},[480,35262,724],{},[480,35264,1166],{},[850,35266,1201],{"stretchy":1186},[467,35268,35269,35301],{},[464,35270,35271,35273,35279,35281,35289,35291,35299],{},[480,35272,14742],{"mathvariant":1206},[477,35274,35275,35277],{},[480,35276,5974],{},[480,35278,14413],{},[850,35280,1187],{"stretchy":1186},[2209,35282,35283,35285,35287],{},[480,35284,285],{},[480,35286,724],{},[480,35288,1166],{},[480,35290,13303],{"mathvariant":1206},[2209,35292,35293,35295,35297],{},[480,35294,13164],{},[480,35296,724],{},[480,35298,1166],{},[850,35300,1201],{"stretchy":1186},[464,35302,35303,35309,35311,35319,35321,35329],{},[477,35304,35305,35307],{},[480,35306,5974],{},[480,35308,14413],{},[850,35310,1187],{"stretchy":1186},[2209,35312,35313,35315,35317],{},[480,35314,285],{},[480,35316,724],{},[480,35318,1166],{},[480,35320,13303],{"mathvariant":1206},[2209,35322,35323,35325,35327],{},[480,35324,13164],{},[480,35326,724],{},[480,35328,1166],{},[850,35330,1201],{"stretchy":1186},[14730,35332,35333,35339],{},[14733,35334,35335],{},[14736,35336,35337],{"scriptlevel":2141,"displaystyle":495},[464,35338],{},[14733,35340,35341],{},[14736,35342,35343],{"scriptlevel":2141,"displaystyle":495},[464,35344,35345,35347,35349,35355,35369,35387,35405,35407,35415,35417,35425,35427],{},[464,35346],{},[850,35348,82],{},[467,35350,35351,35353],{},[470,35352,472],{},[480,35354,7692],{},[15045,35356,35357,35359,35367],{},[850,35358,2213],{},[464,35360,35361,35363,35365],{},[480,35362,724],{},[850,35364,82],{},[470,35366,472],{},[480,35368,7692],{},[15045,35370,35371,35373,35381],{},[850,35372,2213],{},[464,35374,35375,35377,35379],{},[480,35376,1166],{},[850,35378,82],{},[470,35380,472],{},[477,35382,35383,35385],{},[480,35384,13604],{},[480,35386,724],{},[2209,35388,35389,35391,35397],{},[480,35390,1163],{},[854,35392,35393,35395],{},[480,35394,14413],{},[850,35396,19346],{"mathvariant":1206,"lspace":14728,"rspace":14728},[464,35398,35399,35401,35403],{},[480,35400,27872],{},[480,35402,1163],{},[480,35404,5977],{},[850,35406,1187],{"stretchy":1186},[2209,35408,35409,35411,35413],{},[480,35410,13164],{},[480,35412,724],{},[480,35414,1166],{},[850,35416,1192],{"separator":495},[2209,35418,35419,35421,35423],{},[480,35420,285],{},[480,35422,724],{},[480,35424,1166],{},[850,35426,1201],{"stretchy":1186},[467,35428,35429,35461],{},[464,35430,35431,35433,35439,35441,35449,35451,35459],{},[480,35432,14742],{"mathvariant":1206},[477,35434,35435,35437],{},[480,35436,5974],{},[480,35438,14413],{},[850,35440,1187],{"stretchy":1186},[2209,35442,35443,35445,35447],{},[480,35444,285],{},[480,35446,724],{},[480,35448,1166],{},[480,35450,13303],{"mathvariant":1206},[2209,35452,35453,35455,35457],{},[480,35454,13164],{},[480,35456,724],{},[480,35458,1166],{},[850,35460,1201],{"stretchy":1186},[464,35462,35463,35473,35475,35483,35485,35493],{},[477,35464,35465,35467],{},[480,35466,5974],{},[854,35468,35469,35471],{},[480,35470,14413],{},[850,35472,19346],{"mathvariant":1206,"lspace":14728,"rspace":14728},[850,35474,1187],{"stretchy":1186},[2209,35476,35477,35479,35481],{},[480,35478,285],{},[480,35480,724],{},[480,35482,1166],{},[480,35484,13303],{"mathvariant":1206},[2209,35486,35487,35489,35491],{},[480,35488,13164],{},[480,35490,724],{},[480,35492,1166],{},[850,35494,1201],{"stretchy":1186},[487,35496,35497],{"encoding":489},"\\begin{aligned}\n&\\frac{1}{N} \\sum_{n=1}^{N} \\sum_{t=1}^{T_n} A_\\theta^{GAE}(s_n^t, a_n^t) \\nabla \\log P_\\theta(a_n^t | s_n^t) \\\\\n&= \\frac{1}{N} \\sum_{n=1}^{N} \\sum_{t=1}^{T_n} A_{\\theta'}^{GAE}(s_n^t, a_n^t) \\frac{P_\\theta(a_n^t | s_n^t)}{P_{\\theta'}(a_n^t | s_n^t)} \\nabla \\log P_\\theta(a_n^t | s_n^t) \\\\\n&= \\frac{1}{N} \\sum_{n=1}^{N} \\sum_{t=1}^{T_n} A_{\\theta'}^{GAE}(s_n^t, a_n^t) \\frac{P_\\theta(a_n^t | s_n^t)}{P_{\\theta'}(a_n^t | s_n^t)} \\frac{\\nabla P_\\theta(a_n^t | s_n^t)}{P_\\theta(a_n^t | s_n^t)} \\\\\n&= \\frac{1}{N} \\sum_{n=1}^{N} \\sum_{t=1}^{T_n} A_{\\theta'}^{GAE}(s_n^t, a_n^t) \\frac{\\nabla P_\\theta(a_n^t | s_n^t)}{P_{\\theta'}(a_n^t | s_n^t)}\n\\end{aligned}",[47,35499,35501],{"className":35500,"ariaHidden":495},[494],[47,35502,35504,35508],{"className":35503},[499],[47,35505],{"className":35506,"style":35507},[503],"height:13.6262em;vertical-align:-6.5631em;",[47,35509,35511],{"className":35510},[508],[47,35512,35514,35575],{"className":35513},[14724],[47,35515,35517],{"className":35516},[15515],[47,35518,35520,35566],{"className":35519},[520,521],[47,35521,35523,35563],{"className":35522},[525],[47,35524,35527,35536,35545,35554],{"className":35525,"style":35526},[529],"height:7.0631em;",[47,35528,35530,35533],{"style":35529},"top:-9.0631em;",[47,35531],{"className":35532,"style":15532},[537],[47,35534],{"className":35535},[508],[47,35537,35539,35542],{"style":35538},"top:-5.6565em;",[47,35540],{"className":35541,"style":15532},[537],[47,35543],{"className":35544},[508],[47,35546,35548,35551],{"style":35547},"top:-2.25em;",[47,35549],{"className":35550,"style":15532},[537],[47,35552],{"className":35553},[508],[47,35555,35557,35560],{"style":35556},"top:1.1565em;",[47,35558],{"className":35559,"style":15532},[537],[47,35561],{"className":35562},[508],[47,35564,616],{"className":35565},[615],[47,35567,35569],{"className":35568},[525],[47,35570,35573],{"className":35571,"style":35572},[529],"height:6.5631em;",[47,35574],{},[47,35576,35578],{"className":35577},[15739],[47,35579,35581,39287],{"className":35580},[520,521],[47,35582,35584,39284],{"className":35583},[525],[47,35585,35587,36179,37204,38423],{"className":35586,"style":35526},[529],[47,35588,35589,35592],{"style":35529},[47,35590],{"className":35591,"style":15532},[537],[47,35593,35595,35598,35660,35663,35730,35733,35837,35840,35900,35903,35954,35957,35960,36011,36014,36017,36020,36025,36028,36068,36071,36122,36125,36176],{"className":35594},[508],[47,35596],{"className":35597},[508],[47,35599,35601,35604,35657],{"className":35600},[508],[47,35602],{"className":35603},[512,513],[47,35605,35607],{"className":35606},[467],[47,35608,35610,35649],{"className":35609},[520,521],[47,35611,35613,35646],{"className":35612},[525],[47,35614,35616,35627,35635],{"className":35615,"style":16641},[529],[47,35617,35618,35621],{"style":16181},[47,35619],{"className":35620,"style":538},[537],[47,35622,35624],{"className":35623},[508],[47,35625,7692],{"className":35626,"style":7767},[508,581],[47,35628,35629,35632],{"style":663},[47,35630],{"className":35631,"style":538},[537],[47,35633],{"className":35634,"style":671},[670],[47,35636,35637,35640],{"style":16247},[47,35638],{"className":35639,"style":538},[537],[47,35641,35643],{"className":35642},[508],[47,35644,472],{"className":35645},[508],[47,35647,616],{"className":35648},[615],[47,35650,35652],{"className":35651},[525],[47,35653,35655],{"className":35654,"style":16681},[529],[47,35656],{},[47,35658],{"className":35659},[702,513],[47,35661],{"className":35662,"style":1303},[888],[47,35664,35666],{"className":35665},[1338,14600],[47,35667,35669,35722],{"className":35668},[520,521],[47,35670,35672,35719],{"className":35671},[525],[47,35673,35675,35695,35705],{"className":35674,"style":16702},[529],[47,35676,35677,35680],{"style":16705},[47,35678],{"className":35679,"style":15794},[537],[47,35681,35683],{"className":35682},[542,543,544,545],[47,35684,35686,35689,35692],{"className":35685},[508,545],[47,35687,724],{"className":35688},[508,581,545],[47,35690,82],{"className":35691},[893,545],[47,35693,472],{"className":35694},[508,545],[47,35696,35697,35700],{"style":15803},[47,35698],{"className":35699,"style":15794},[537],[47,35701,35702],{},[47,35703,2213],{"className":35704},[1338,2297,15812],[47,35706,35707,35710],{"style":16736},[47,35708],{"className":35709,"style":15794},[537],[47,35711,35713],{"className":35712},[542,543,544,545],[47,35714,35716],{"className":35715},[508,545],[47,35717,7692],{"className":35718,"style":7767},[508,581,545],[47,35720,616],{"className":35721},[615],[47,35723,35725],{"className":35724},[525],[47,35726,35728],{"className":35727,"style":16755},[529],[47,35729],{},[47,35731],{"className":35732,"style":1303},[888],[47,35734,35736],{"className":35735},[1338,14600],[47,35737,35739,35829],{"className":35738},[520,521],[47,35740,35742,35826],{"className":35741},[525],[47,35743,35745,35765,35775],{"className":35744,"style":17511},[529],[47,35746,35747,35750],{"style":16705},[47,35748],{"className":35749,"style":15794},[537],[47,35751,35753],{"className":35752},[542,543,544,545],[47,35754,35756,35759,35762],{"className":35755},[508,545],[47,35757,1166],{"className":35758},[508,581,545],[47,35760,82],{"className":35761},[893,545],[47,35763,472],{"className":35764},[508,545],[47,35766,35767,35770],{"style":15803},[47,35768],{"className":35769,"style":15794},[537],[47,35771,35772],{},[47,35773,2213],{"className":35774},[1338,2297,15812],[47,35776,35777,35780],{"style":17544},[47,35778],{"className":35779,"style":15794},[537],[47,35781,35783],{"className":35782},[542,543,544,545],[47,35784,35786],{"className":35785},[508,545],[47,35787,35789,35792],{"className":35788},[508,545],[47,35790,13604],{"className":35791,"style":3555},[508,581,545],[47,35793,35795],{"className":35794},[585],[47,35796,35798,35818],{"className":35797},[520,521],[47,35799,35801,35815],{"className":35800},[525],[47,35802,35804],{"className":35803,"style":4672},[529],[47,35805,35806,35809],{"style":17574},[47,35807],{"className":35808,"style":602},[537],[47,35810,35812],{"className":35811},[542,606,607,545],[47,35813,724],{"className":35814},[508,581,545],[47,35816,616],{"className":35817},[615],[47,35819,35821],{"className":35820},[525],[47,35822,35824],{"className":35823,"style":4694},[529],[47,35825],{},[47,35827,616],{"className":35828},[615],[47,35830,35832],{"className":35831},[525],[47,35833,35835],{"className":35834,"style":16755},[529],[47,35836],{},[47,35838],{"className":35839,"style":1303},[888],[47,35841,35843,35846],{"className":35842},[508],[47,35844,1163],{"className":35845},[508,581],[47,35847,35849],{"className":35848},[585],[47,35850,35852,35892],{"className":35851},[520,521],[47,35853,35855,35889],{"className":35854},[525],[47,35856,35858,35869],{"className":35857,"style":25158},[529],[47,35859,35860,35863],{"style":17673},[47,35861],{"className":35862,"style":929},[537],[47,35864,35866],{"className":35865},[542,543,544,545],[47,35867,14413],{"className":35868,"style":3580},[508,581,545],[47,35870,35871,35874],{"style":16788},[47,35872],{"className":35873,"style":929},[537],[47,35875,35877],{"className":35876},[542,543,544,545],[47,35878,35880,35883,35886],{"className":35879},[508,545],[47,35881,27872],{"className":35882},[508,581,545],[47,35884,1163],{"className":35885},[508,581,545],[47,35887,5977],{"className":35888,"style":6420},[508,581,545],[47,35890,616],{"className":35891},[615],[47,35893,35895],{"className":35894},[525],[47,35896,35898],{"className":35897,"style":17703},[529],[47,35899],{},[47,35901,1187],{"className":35902},[512],[47,35904,35906,35909],{"className":35905},[508],[47,35907,13164],{"className":35908},[508,581],[47,35910,35912],{"className":35911},[585],[47,35913,35915,35946],{"className":35914},[520,521],[47,35916,35918,35943],{"className":35917},[525],[47,35919,35921,35932],{"className":35920,"style":17670},[529],[47,35922,35923,35926],{"style":17673},[47,35924],{"className":35925,"style":929},[537],[47,35927,35929],{"className":35928},[542,543,544,545],[47,35930,724],{"className":35931},[508,581,545],[47,35933,35934,35937],{"style":16788},[47,35935],{"className":35936,"style":929},[537],[47,35938,35940],{"className":35939},[542,543,544,545],[47,35941,1166],{"className":35942},[508,581,545],[47,35944,616],{"className":35945},[615],[47,35947,35949],{"className":35948},[525],[47,35950,35952],{"className":35951,"style":17703},[529],[47,35953],{},[47,35955,1192],{"className":35956},[1299],[47,35958],{"className":35959,"style":1303},[888],[47,35961,35963,35966],{"className":35962},[508],[47,35964,285],{"className":35965},[508,581],[47,35967,35969],{"className":35968},[585],[47,35970,35972,36003],{"className":35971},[520,521],[47,35973,35975,36000],{"className":35974},[525],[47,35976,35978,35989],{"className":35977,"style":17670},[529],[47,35979,35980,35983],{"style":17673},[47,35981],{"className":35982,"style":929},[537],[47,35984,35986],{"className":35985},[542,543,544,545],[47,35987,724],{"className":35988},[508,581,545],[47,35990,35991,35994],{"style":16788},[47,35992],{"className":35993,"style":929},[537],[47,35995,35997],{"className":35996},[542,543,544,545],[47,35998,1166],{"className":35999},[508,581,545],[47,36001,616],{"className":36002},[615],[47,36004,36006],{"className":36005},[525],[47,36007,36009],{"className":36008,"style":17703},[529],[47,36010],{},[47,36012,1201],{"className":36013},[702],[47,36015,14742],{"className":36016},[508],[47,36018],{"className":36019,"style":1303},[888],[47,36021,17212,36023],{"className":36022},[1338],[47,36024,17216],{"style":17215},[47,36026],{"className":36027,"style":1303},[888],[47,36029,36031,36034],{"className":36030},[508],[47,36032,5974],{"className":36033,"style":3555},[508,581],[47,36035,36037],{"className":36036},[585],[47,36038,36040,36060],{"className":36039},[520,521],[47,36041,36043,36057],{"className":36042},[525],[47,36044,36046],{"className":36045,"style":1635},[529],[47,36047,36048,36051],{"style":7830},[47,36049],{"className":36050,"style":929},[537],[47,36052,36054],{"className":36053},[542,543,544,545],[47,36055,14413],{"className":36056,"style":3580},[508,581,545],[47,36058,616],{"className":36059},[615],[47,36061,36063],{"className":36062},[525],[47,36064,36066],{"className":36065,"style":1657},[529],[47,36067],{},[47,36069,1187],{"className":36070},[512],[47,36072,36074,36077],{"className":36073},[508],[47,36075,285],{"className":36076},[508,581],[47,36078,36080],{"className":36079},[585],[47,36081,36083,36114],{"className":36082},[520,521],[47,36084,36086,36111],{"className":36085},[525],[47,36087,36089,36100],{"className":36088,"style":17670},[529],[47,36090,36091,36094],{"style":17673},[47,36092],{"className":36093,"style":929},[537],[47,36095,36097],{"className":36096},[542,543,544,545],[47,36098,724],{"className":36099},[508,581,545],[47,36101,36102,36105],{"style":16788},[47,36103],{"className":36104,"style":929},[537],[47,36106,36108],{"className":36107},[542,543,544,545],[47,36109,1166],{"className":36110},[508,581,545],[47,36112,616],{"className":36113},[615],[47,36115,36117],{"className":36116},[525],[47,36118,36120],{"className":36119,"style":17703},[529],[47,36121],{},[47,36123,13303],{"className":36124},[508],[47,36126,36128,36131],{"className":36127},[508],[47,36129,13164],{"className":36130},[508,581],[47,36132,36134],{"className":36133},[585],[47,36135,36137,36168],{"className":36136},[520,521],[47,36138,36140,36165],{"className":36139},[525],[47,36141,36143,36154],{"className":36142,"style":17670},[529],[47,36144,36145,36148],{"style":17673},[47,36146],{"className":36147,"style":929},[537],[47,36149,36151],{"className":36150},[542,543,544,545],[47,36152,724],{"className":36153},[508,581,545],[47,36155,36156,36159],{"style":16788},[47,36157],{"className":36158,"style":929},[537],[47,36160,36162],{"className":36161},[542,543,544,545],[47,36163,1166],{"className":36164},[508,581,545],[47,36166,616],{"className":36167},[615],[47,36169,36171],{"className":36170},[525],[47,36172,36174],{"className":36173,"style":17703},[529],[47,36175],{},[47,36177,1201],{"className":36178},[702],[47,36180,36181,36184],{"style":35538},[47,36182],{"className":36183,"style":15532},[537],[47,36185,36187,36190,36193,36196,36199,36261,36264,36331,36334,36438,36441,36533,36536,36587,36590,36593,36644,36647,37039,37042,37045,37050,37053,37093,37096,37147,37150,37201],{"className":36186},[508],[47,36188],{"className":36189},[508],[47,36191],{"className":36192,"style":889},[888],[47,36194,82],{"className":36195},[893],[47,36197],{"className":36198,"style":889},[888],[47,36200,36202,36205,36258],{"className":36201},[508],[47,36203],{"className":36204},[512,513],[47,36206,36208],{"className":36207},[467],[47,36209,36211,36250],{"className":36210},[520,521],[47,36212,36214,36247],{"className":36213},[525],[47,36215,36217,36228,36236],{"className":36216,"style":16641},[529],[47,36218,36219,36222],{"style":16181},[47,36220],{"className":36221,"style":538},[537],[47,36223,36225],{"className":36224},[508],[47,36226,7692],{"className":36227,"style":7767},[508,581],[47,36229,36230,36233],{"style":663},[47,36231],{"className":36232,"style":538},[537],[47,36234],{"className":36235,"style":671},[670],[47,36237,36238,36241],{"style":16247},[47,36239],{"className":36240,"style":538},[537],[47,36242,36244],{"className":36243},[508],[47,36245,472],{"className":36246},[508],[47,36248,616],{"className":36249},[615],[47,36251,36253],{"className":36252},[525],[47,36254,36256],{"className":36255,"style":16681},[529],[47,36257],{},[47,36259],{"className":36260},[702,513],[47,36262],{"className":36263,"style":1303},[888],[47,36265,36267],{"className":36266},[1338,14600],[47,36268,36270,36323],{"className":36269},[520,521],[47,36271,36273,36320],{"className":36272},[525],[47,36274,36276,36296,36306],{"className":36275,"style":16702},[529],[47,36277,36278,36281],{"style":16705},[47,36279],{"className":36280,"style":15794},[537],[47,36282,36284],{"className":36283},[542,543,544,545],[47,36285,36287,36290,36293],{"className":36286},[508,545],[47,36288,724],{"className":36289},[508,581,545],[47,36291,82],{"className":36292},[893,545],[47,36294,472],{"className":36295},[508,545],[47,36297,36298,36301],{"style":15803},[47,36299],{"className":36300,"style":15794},[537],[47,36302,36303],{},[47,36304,2213],{"className":36305},[1338,2297,15812],[47,36307,36308,36311],{"style":16736},[47,36309],{"className":36310,"style":15794},[537],[47,36312,36314],{"className":36313},[542,543,544,545],[47,36315,36317],{"className":36316},[508,545],[47,36318,7692],{"className":36319,"style":7767},[508,581,545],[47,36321,616],{"className":36322},[615],[47,36324,36326],{"className":36325},[525],[47,36327,36329],{"className":36328,"style":16755},[529],[47,36330],{},[47,36332],{"className":36333,"style":1303},[888],[47,36335,36337],{"className":36336},[1338,14600],[47,36338,36340,36430],{"className":36339},[520,521],[47,36341,36343,36427],{"className":36342},[525],[47,36344,36346,36366,36376],{"className":36345,"style":17511},[529],[47,36347,36348,36351],{"style":16705},[47,36349],{"className":36350,"style":15794},[537],[47,36352,36354],{"className":36353},[542,543,544,545],[47,36355,36357,36360,36363],{"className":36356},[508,545],[47,36358,1166],{"className":36359},[508,581,545],[47,36361,82],{"className":36362},[893,545],[47,36364,472],{"className":36365},[508,545],[47,36367,36368,36371],{"style":15803},[47,36369],{"className":36370,"style":15794},[537],[47,36372,36373],{},[47,36374,2213],{"className":36375},[1338,2297,15812],[47,36377,36378,36381],{"style":17544},[47,36379],{"className":36380,"style":15794},[537],[47,36382,36384],{"className":36383},[542,543,544,545],[47,36385,36387],{"className":36386},[508,545],[47,36388,36390,36393],{"className":36389},[508,545],[47,36391,13604],{"className":36392,"style":3555},[508,581,545],[47,36394,36396],{"className":36395},[585],[47,36397,36399,36419],{"className":36398},[520,521],[47,36400,36402,36416],{"className":36401},[525],[47,36403,36405],{"className":36404,"style":4672},[529],[47,36406,36407,36410],{"style":17574},[47,36408],{"className":36409,"style":602},[537],[47,36411,36413],{"className":36412},[542,606,607,545],[47,36414,724],{"className":36415},[508,581,545],[47,36417,616],{"className":36418},[615],[47,36420,36422],{"className":36421},[525],[47,36423,36425],{"className":36424,"style":4694},[529],[47,36426],{},[47,36428,616],{"className":36429},[615],[47,36431,36433],{"className":36432},[525],[47,36434,36436],{"className":36435,"style":16755},[529],[47,36437],{},[47,36439],{"className":36440,"style":1303},[888],[47,36442,36444,36447],{"className":36443},[508],[47,36445,1163],{"className":36446},[508,581],[47,36448,36450],{"className":36449},[585],[47,36451,36453,36525],{"className":36452},[520,521],[47,36454,36456,36522],{"className":36455},[525],[47,36457,36459,36502],{"className":36458,"style":25158},[529],[47,36460,36461,36464],{"style":17673},[47,36462],{"className":36463,"style":929},[537],[47,36465,36467],{"className":36466},[542,543,544,545],[47,36468,36470],{"className":36469},[508,545],[47,36471,36473,36476],{"className":36472},[508,545],[47,36474,14413],{"className":36475,"style":3580},[508,581,545],[47,36477,36479],{"className":36478},[585],[47,36480,36482],{"className":36481},[520],[47,36483,36485],{"className":36484},[525],[47,36486,36488],{"className":36487,"style":19506},[529],[47,36489,36490,36493],{"style":19509},[47,36491],{"className":36492,"style":602},[537],[47,36494,36496],{"className":36495},[542,606,607,545],[47,36497,36499],{"className":36498},[508,545],[47,36500,19346],{"className":36501},[508,545],[47,36503,36504,36507],{"style":16788},[47,36505],{"className":36506,"style":929},[537],[47,36508,36510],{"className":36509},[542,543,544,545],[47,36511,36513,36516,36519],{"className":36512},[508,545],[47,36514,27872],{"className":36515},[508,581,545],[47,36517,1163],{"className":36518},[508,581,545],[47,36520,5977],{"className":36521,"style":6420},[508,581,545],[47,36523,616],{"className":36524},[615],[47,36526,36528],{"className":36527},[525],[47,36529,36531],{"className":36530,"style":17703},[529],[47,36532],{},[47,36534,1187],{"className":36535},[512],[47,36537,36539,36542],{"className":36538},[508],[47,36540,13164],{"className":36541},[508,581],[47,36543,36545],{"className":36544},[585],[47,36546,36548,36579],{"className":36547},[520,521],[47,36549,36551,36576],{"className":36550},[525],[47,36552,36554,36565],{"className":36553,"style":17670},[529],[47,36555,36556,36559],{"style":17673},[47,36557],{"className":36558,"style":929},[537],[47,36560,36562],{"className":36561},[542,543,544,545],[47,36563,724],{"className":36564},[508,581,545],[47,36566,36567,36570],{"style":16788},[47,36568],{"className":36569,"style":929},[537],[47,36571,36573],{"className":36572},[542,543,544,545],[47,36574,1166],{"className":36575},[508,581,545],[47,36577,616],{"className":36578},[615],[47,36580,36582],{"className":36581},[525],[47,36583,36585],{"className":36584,"style":17703},[529],[47,36586],{},[47,36588,1192],{"className":36589},[1299],[47,36591],{"className":36592,"style":1303},[888],[47,36594,36596,36599],{"className":36595},[508],[47,36597,285],{"className":36598},[508,581],[47,36600,36602],{"className":36601},[585],[47,36603,36605,36636],{"className":36604},[520,521],[47,36606,36608,36633],{"className":36607},[525],[47,36609,36611,36622],{"className":36610,"style":17670},[529],[47,36612,36613,36616],{"style":17673},[47,36614],{"className":36615,"style":929},[537],[47,36617,36619],{"className":36618},[542,543,544,545],[47,36620,724],{"className":36621},[508,581,545],[47,36623,36624,36627],{"style":16788},[47,36625],{"className":36626,"style":929},[537],[47,36628,36630],{"className":36629},[542,543,544,545],[47,36631,1166],{"className":36632},[508,581,545],[47,36634,616],{"className":36635},[615],[47,36637,36639],{"className":36638},[525],[47,36640,36642],{"className":36641,"style":17703},[529],[47,36643],{},[47,36645,1201],{"className":36646},[702],[47,36648,36650,36653,37036],{"className":36649},[508],[47,36651],{"className":36652},[512,513],[47,36654,36656],{"className":36655},[467],[47,36657,36659,37028],{"className":36658},[520,521],[47,36660,36662,37025],{"className":36661},[525],[47,36663,36666,36858,36866],{"className":36664,"style":36665},[529],"height:1.4706em;",[47,36667,36668,36671],{"style":16181},[47,36669],{"className":36670,"style":538},[537],[47,36672,36674,36746,36749,36801,36804,36855],{"className":36673},[508],[47,36675,36677,36680],{"className":36676},[508],[47,36678,5974],{"className":36679,"style":3555},[508,581],[47,36681,36683],{"className":36682},[585],[47,36684,36686,36738],{"className":36685},[520,521],[47,36687,36689,36735],{"className":36688},[525],[47,36690,36692],{"className":36691,"style":1635},[529],[47,36693,36694,36697],{"style":7830},[47,36695],{"className":36696,"style":929},[537],[47,36698,36700],{"className":36699},[542,543,544,545],[47,36701,36703],{"className":36702},[508,545],[47,36704,36706,36709],{"className":36705},[508,545],[47,36707,14413],{"className":36708,"style":3580},[508,581,545],[47,36710,36712],{"className":36711},[585],[47,36713,36715],{"className":36714},[520],[47,36716,36718],{"className":36717},[525],[47,36719,36721],{"className":36720,"style":19506},[529],[47,36722,36723,36726],{"style":19509},[47,36724],{"className":36725,"style":602},[537],[47,36727,36729],{"className":36728},[542,606,607,545],[47,36730,36732],{"className":36731},[508,545],[47,36733,19346],{"className":36734},[508,545],[47,36736,616],{"className":36737},[615],[47,36739,36741],{"className":36740},[525],[47,36742,36744],{"className":36743,"style":1657},[529],[47,36745],{},[47,36747,1187],{"className":36748},[512],[47,36750,36752,36755],{"className":36751},[508],[47,36753,285],{"className":36754},[508,581],[47,36756,36758],{"className":36757},[585],[47,36759,36761,36793],{"className":36760},[520,521],[47,36762,36764,36790],{"className":36763},[525],[47,36765,36768,36779],{"className":36766,"style":36767},[529],"height:0.7196em;",[47,36769,36770,36773],{"style":17673},[47,36771],{"className":36772,"style":929},[537],[47,36774,36776],{"className":36775},[542,543,544,545],[47,36777,724],{"className":36778},[508,581,545],[47,36780,36781,36784],{"style":16891},[47,36782],{"className":36783,"style":929},[537],[47,36785,36787],{"className":36786},[542,543,544,545],[47,36788,1166],{"className":36789},[508,581,545],[47,36791,616],{"className":36792},[615],[47,36794,36796],{"className":36795},[525],[47,36797,36799],{"className":36798,"style":17703},[529],[47,36800],{},[47,36802,13303],{"className":36803},[508],[47,36805,36807,36810],{"className":36806},[508],[47,36808,13164],{"className":36809},[508,581],[47,36811,36813],{"className":36812},[585],[47,36814,36816,36847],{"className":36815},[520,521],[47,36817,36819,36844],{"className":36818},[525],[47,36820,36822,36833],{"className":36821,"style":36767},[529],[47,36823,36824,36827],{"style":17673},[47,36825],{"className":36826,"style":929},[537],[47,36828,36830],{"className":36829},[542,543,544,545],[47,36831,724],{"className":36832},[508,581,545],[47,36834,36835,36838],{"style":16891},[47,36836],{"className":36837,"style":929},[537],[47,36839,36841],{"className":36840},[542,543,544,545],[47,36842,1166],{"className":36843},[508,581,545],[47,36845,616],{"className":36846},[615],[47,36848,36850],{"className":36849},[525],[47,36851,36853],{"className":36852,"style":17703},[529],[47,36854],{},[47,36856,1201],{"className":36857},[702],[47,36859,36860,36863],{"style":663},[47,36861],{"className":36862,"style":538},[537],[47,36864],{"className":36865,"style":671},[670],[47,36867,36868,36871],{"style":16247},[47,36869],{"className":36870,"style":538},[537],[47,36872,36874,36914,36917,36968,36971,37022],{"className":36873},[508],[47,36875,36877,36880],{"className":36876},[508],[47,36878,5974],{"className":36879,"style":3555},[508,581],[47,36881,36883],{"className":36882},[585],[47,36884,36886,36906],{"className":36885},[520,521],[47,36887,36889,36903],{"className":36888},[525],[47,36890,36892],{"className":36891,"style":1635},[529],[47,36893,36894,36897],{"style":7830},[47,36895],{"className":36896,"style":929},[537],[47,36898,36900],{"className":36899},[542,543,544,545],[47,36901,14413],{"className":36902,"style":3580},[508,581,545],[47,36904,616],{"className":36905},[615],[47,36907,36909],{"className":36908},[525],[47,36910,36912],{"className":36911,"style":1657},[529],[47,36913],{},[47,36915,1187],{"className":36916},[512],[47,36918,36920,36923],{"className":36919},[508],[47,36921,285],{"className":36922},[508,581],[47,36924,36926],{"className":36925},[585],[47,36927,36929,36960],{"className":36928},[520,521],[47,36930,36932,36957],{"className":36931},[525],[47,36933,36935,36946],{"className":36934,"style":19211},[529],[47,36936,36937,36940],{"style":17673},[47,36938],{"className":36939,"style":929},[537],[47,36941,36943],{"className":36942},[542,543,544,545],[47,36944,724],{"className":36945},[508,581,545],[47,36947,36948,36951],{"style":925},[47,36949],{"className":36950,"style":929},[537],[47,36952,36954],{"className":36953},[542,543,544,545],[47,36955,1166],{"className":36956},[508,581,545],[47,36958,616],{"className":36959},[615],[47,36961,36963],{"className":36962},[525],[47,36964,36966],{"className":36965,"style":17703},[529],[47,36967],{},[47,36969,13303],{"className":36970},[508],[47,36972,36974,36977],{"className":36973},[508],[47,36975,13164],{"className":36976},[508,581],[47,36978,36980],{"className":36979},[585],[47,36981,36983,37014],{"className":36982},[520,521],[47,36984,36986,37011],{"className":36985},[525],[47,36987,36989,37000],{"className":36988,"style":19211},[529],[47,36990,36991,36994],{"style":17673},[47,36992],{"className":36993,"style":929},[537],[47,36995,36997],{"className":36996},[542,543,544,545],[47,36998,724],{"className":36999},[508,581,545],[47,37001,37002,37005],{"style":925},[47,37003],{"className":37004,"style":929},[537],[47,37006,37008],{"className":37007},[542,543,544,545],[47,37009,1166],{"className":37010},[508,581,545],[47,37012,616],{"className":37013},[615],[47,37015,37017],{"className":37016},[525],[47,37018,37020],{"className":37019,"style":17703},[529],[47,37021],{},[47,37023,1201],{"className":37024},[702],[47,37026,616],{"className":37027},[615],[47,37029,37031],{"className":37030},[525],[47,37032,37034],{"className":37033,"style":16312},[529],[47,37035],{},[47,37037],{"className":37038},[702,513],[47,37040,14742],{"className":37041},[508],[47,37043],{"className":37044,"style":1303},[888],[47,37046,17212,37048],{"className":37047},[1338],[47,37049,17216],{"style":17215},[47,37051],{"className":37052,"style":1303},[888],[47,37054,37056,37059],{"className":37055},[508],[47,37057,5974],{"className":37058,"style":3555},[508,581],[47,37060,37062],{"className":37061},[585],[47,37063,37065,37085],{"className":37064},[520,521],[47,37066,37068,37082],{"className":37067},[525],[47,37069,37071],{"className":37070,"style":1635},[529],[47,37072,37073,37076],{"style":7830},[47,37074],{"className":37075,"style":929},[537],[47,37077,37079],{"className":37078},[542,543,544,545],[47,37080,14413],{"className":37081,"style":3580},[508,581,545],[47,37083,616],{"className":37084},[615],[47,37086,37088],{"className":37087},[525],[47,37089,37091],{"className":37090,"style":1657},[529],[47,37092],{},[47,37094,1187],{"className":37095},[512],[47,37097,37099,37102],{"className":37098},[508],[47,37100,285],{"className":37101},[508,581],[47,37103,37105],{"className":37104},[585],[47,37106,37108,37139],{"className":37107},[520,521],[47,37109,37111,37136],{"className":37110},[525],[47,37112,37114,37125],{"className":37113,"style":17670},[529],[47,37115,37116,37119],{"style":17673},[47,37117],{"className":37118,"style":929},[537],[47,37120,37122],{"className":37121},[542,543,544,545],[47,37123,724],{"className":37124},[508,581,545],[47,37126,37127,37130],{"style":16788},[47,37128],{"className":37129,"style":929},[537],[47,37131,37133],{"className":37132},[542,543,544,545],[47,37134,1166],{"className":37135},[508,581,545],[47,37137,616],{"className":37138},[615],[47,37140,37142],{"className":37141},[525],[47,37143,37145],{"className":37144,"style":17703},[529],[47,37146],{},[47,37148,13303],{"className":37149},[508],[47,37151,37153,37156],{"className":37152},[508],[47,37154,13164],{"className":37155},[508,581],[47,37157,37159],{"className":37158},[585],[47,37160,37162,37193],{"className":37161},[520,521],[47,37163,37165,37190],{"className":37164},[525],[47,37166,37168,37179],{"className":37167,"style":17670},[529],[47,37169,37170,37173],{"style":17673},[47,37171],{"className":37172,"style":929},[537],[47,37174,37176],{"className":37175},[542,543,544,545],[47,37177,724],{"className":37178},[508,581,545],[47,37180,37181,37184],{"style":16788},[47,37182],{"className":37183,"style":929},[537],[47,37185,37187],{"className":37186},[542,543,544,545],[47,37188,1166],{"className":37189},[508,581,545],[47,37191,616],{"className":37192},[615],[47,37194,37196],{"className":37195},[525],[47,37197,37199],{"className":37198,"style":17703},[529],[47,37200],{},[47,37202,1201],{"className":37203},[702],[47,37205,37206,37209],{"style":35547},[47,37207],{"className":37208,"style":15532},[537],[47,37210,37212,37215,37218,37221,37224,37286,37289,37356,37359,37463,37466,37558,37561,37612,37615,37618,37669,37672,38062],{"className":37211},[508],[47,37213],{"className":37214},[508],[47,37216],{"className":37217,"style":889},[888],[47,37219,82],{"className":37220},[893],[47,37222],{"className":37223,"style":889},[888],[47,37225,37227,37230,37283],{"className":37226},[508],[47,37228],{"className":37229},[512,513],[47,37231,37233],{"className":37232},[467],[47,37234,37236,37275],{"className":37235},[520,521],[47,37237,37239,37272],{"className":37238},[525],[47,37240,37242,37253,37261],{"className":37241,"style":16641},[529],[47,37243,37244,37247],{"style":16181},[47,37245],{"className":37246,"style":538},[537],[47,37248,37250],{"className":37249},[508],[47,37251,7692],{"className":37252,"style":7767},[508,581],[47,37254,37255,37258],{"style":663},[47,37256],{"className":37257,"style":538},[537],[47,37259],{"className":37260,"style":671},[670],[47,37262,37263,37266],{"style":16247},[47,37264],{"className":37265,"style":538},[537],[47,37267,37269],{"className":37268},[508],[47,37270,472],{"className":37271},[508],[47,37273,616],{"className":37274},[615],[47,37276,37278],{"className":37277},[525],[47,37279,37281],{"className":37280,"style":16681},[529],[47,37282],{},[47,37284],{"className":37285},[702,513],[47,37287],{"className":37288,"style":1303},[888],[47,37290,37292],{"className":37291},[1338,14600],[47,37293,37295,37348],{"className":37294},[520,521],[47,37296,37298,37345],{"className":37297},[525],[47,37299,37301,37321,37331],{"className":37300,"style":16702},[529],[47,37302,37303,37306],{"style":16705},[47,37304],{"className":37305,"style":15794},[537],[47,37307,37309],{"className":37308},[542,543,544,545],[47,37310,37312,37315,37318],{"className":37311},[508,545],[47,37313,724],{"className":37314},[508,581,545],[47,37316,82],{"className":37317},[893,545],[47,37319,472],{"className":37320},[508,545],[47,37322,37323,37326],{"style":15803},[47,37324],{"className":37325,"style":15794},[537],[47,37327,37328],{},[47,37329,2213],{"className":37330},[1338,2297,15812],[47,37332,37333,37336],{"style":16736},[47,37334],{"className":37335,"style":15794},[537],[47,37337,37339],{"className":37338},[542,543,544,545],[47,37340,37342],{"className":37341},[508,545],[47,37343,7692],{"className":37344,"style":7767},[508,581,545],[47,37346,616],{"className":37347},[615],[47,37349,37351],{"className":37350},[525],[47,37352,37354],{"className":37353,"style":16755},[529],[47,37355],{},[47,37357],{"className":37358,"style":1303},[888],[47,37360,37362],{"className":37361},[1338,14600],[47,37363,37365,37455],{"className":37364},[520,521],[47,37366,37368,37452],{"className":37367},[525],[47,37369,37371,37391,37401],{"className":37370,"style":17511},[529],[47,37372,37373,37376],{"style":16705},[47,37374],{"className":37375,"style":15794},[537],[47,37377,37379],{"className":37378},[542,543,544,545],[47,37380,37382,37385,37388],{"className":37381},[508,545],[47,37383,1166],{"className":37384},[508,581,545],[47,37386,82],{"className":37387},[893,545],[47,37389,472],{"className":37390},[508,545],[47,37392,37393,37396],{"style":15803},[47,37394],{"className":37395,"style":15794},[537],[47,37397,37398],{},[47,37399,2213],{"className":37400},[1338,2297,15812],[47,37402,37403,37406],{"style":17544},[47,37404],{"className":37405,"style":15794},[537],[47,37407,37409],{"className":37408},[542,543,544,545],[47,37410,37412],{"className":37411},[508,545],[47,37413,37415,37418],{"className":37414},[508,545],[47,37416,13604],{"className":37417,"style":3555},[508,581,545],[47,37419,37421],{"className":37420},[585],[47,37422,37424,37444],{"className":37423},[520,521],[47,37425,37427,37441],{"className":37426},[525],[47,37428,37430],{"className":37429,"style":4672},[529],[47,37431,37432,37435],{"style":17574},[47,37433],{"className":37434,"style":602},[537],[47,37436,37438],{"className":37437},[542,606,607,545],[47,37439,724],{"className":37440},[508,581,545],[47,37442,616],{"className":37443},[615],[47,37445,37447],{"className":37446},[525],[47,37448,37450],{"className":37449,"style":4694},[529],[47,37451],{},[47,37453,616],{"className":37454},[615],[47,37456,37458],{"className":37457},[525],[47,37459,37461],{"className":37460,"style":16755},[529],[47,37462],{},[47,37464],{"className":37465,"style":1303},[888],[47,37467,37469,37472],{"className":37468},[508],[47,37470,1163],{"className":37471},[508,581],[47,37473,37475],{"className":37474},[585],[47,37476,37478,37550],{"className":37477},[520,521],[47,37479,37481,37547],{"className":37480},[525],[47,37482,37484,37527],{"className":37483,"style":25158},[529],[47,37485,37486,37489],{"style":17673},[47,37487],{"className":37488,"style":929},[537],[47,37490,37492],{"className":37491},[542,543,544,545],[47,37493,37495],{"className":37494},[508,545],[47,37496,37498,37501],{"className":37497},[508,545],[47,37499,14413],{"className":37500,"style":3580},[508,581,545],[47,37502,37504],{"className":37503},[585],[47,37505,37507],{"className":37506},[520],[47,37508,37510],{"className":37509},[525],[47,37511,37513],{"className":37512,"style":19506},[529],[47,37514,37515,37518],{"style":19509},[47,37516],{"className":37517,"style":602},[537],[47,37519,37521],{"className":37520},[542,606,607,545],[47,37522,37524],{"className":37523},[508,545],[47,37525,19346],{"className":37526},[508,545],[47,37528,37529,37532],{"style":16788},[47,37530],{"className":37531,"style":929},[537],[47,37533,37535],{"className":37534},[542,543,544,545],[47,37536,37538,37541,37544],{"className":37537},[508,545],[47,37539,27872],{"className":37540},[508,581,545],[47,37542,1163],{"className":37543},[508,581,545],[47,37545,5977],{"className":37546,"style":6420},[508,581,545],[47,37548,616],{"className":37549},[615],[47,37551,37553],{"className":37552},[525],[47,37554,37556],{"className":37555,"style":17703},[529],[47,37557],{},[47,37559,1187],{"className":37560},[512],[47,37562,37564,37567],{"className":37563},[508],[47,37565,13164],{"className":37566},[508,581],[47,37568,37570],{"className":37569},[585],[47,37571,37573,37604],{"className":37572},[520,521],[47,37574,37576,37601],{"className":37575},[525],[47,37577,37579,37590],{"className":37578,"style":17670},[529],[47,37580,37581,37584],{"style":17673},[47,37582],{"className":37583,"style":929},[537],[47,37585,37587],{"className":37586},[542,543,544,545],[47,37588,724],{"className":37589},[508,581,545],[47,37591,37592,37595],{"style":16788},[47,37593],{"className":37594,"style":929},[537],[47,37596,37598],{"className":37597},[542,543,544,545],[47,37599,1166],{"className":37600},[508,581,545],[47,37602,616],{"className":37603},[615],[47,37605,37607],{"className":37606},[525],[47,37608,37610],{"className":37609,"style":17703},[529],[47,37611],{},[47,37613,1192],{"className":37614},[1299],[47,37616],{"className":37617,"style":1303},[888],[47,37619,37621,37624],{"className":37620},[508],[47,37622,285],{"className":37623},[508,581],[47,37625,37627],{"className":37626},[585],[47,37628,37630,37661],{"className":37629},[520,521],[47,37631,37633,37658],{"className":37632},[525],[47,37634,37636,37647],{"className":37635,"style":17670},[529],[47,37637,37638,37641],{"style":17673},[47,37639],{"className":37640,"style":929},[537],[47,37642,37644],{"className":37643},[542,543,544,545],[47,37645,724],{"className":37646},[508,581,545],[47,37648,37649,37652],{"style":16788},[47,37650],{"className":37651,"style":929},[537],[47,37653,37655],{"className":37654},[542,543,544,545],[47,37656,1166],{"className":37657},[508,581,545],[47,37659,616],{"className":37660},[615],[47,37662,37664],{"className":37663},[525],[47,37665,37667],{"className":37666,"style":17703},[529],[47,37668],{},[47,37670,1201],{"className":37671},[702],[47,37673,37675,37678,38059],{"className":37674},[508],[47,37676],{"className":37677},[512,513],[47,37679,37681],{"className":37680},[467],[47,37682,37684,38051],{"className":37683},[520,521],[47,37685,37687,38048],{"className":37686},[525],[47,37688,37690,37881,37889],{"className":37689,"style":36665},[529],[47,37691,37692,37695],{"style":16181},[47,37693],{"className":37694,"style":538},[537],[47,37696,37698,37770,37773,37824,37827,37878],{"className":37697},[508],[47,37699,37701,37704],{"className":37700},[508],[47,37702,5974],{"className":37703,"style":3555},[508,581],[47,37705,37707],{"className":37706},[585],[47,37708,37710,37762],{"className":37709},[520,521],[47,37711,37713,37759],{"className":37712},[525],[47,37714,37716],{"className":37715,"style":1635},[529],[47,37717,37718,37721],{"style":7830},[47,37719],{"className":37720,"style":929},[537],[47,37722,37724],{"className":37723},[542,543,544,545],[47,37725,37727],{"className":37726},[508,545],[47,37728,37730,37733],{"className":37729},[508,545],[47,37731,14413],{"className":37732,"style":3580},[508,581,545],[47,37734,37736],{"className":37735},[585],[47,37737,37739],{"className":37738},[520],[47,37740,37742],{"className":37741},[525],[47,37743,37745],{"className":37744,"style":19506},[529],[47,37746,37747,37750],{"style":19509},[47,37748],{"className":37749,"style":602},[537],[47,37751,37753],{"className":37752},[542,606,607,545],[47,37754,37756],{"className":37755},[508,545],[47,37757,19346],{"className":37758},[508,545],[47,37760,616],{"className":37761},[615],[47,37763,37765],{"className":37764},[525],[47,37766,37768],{"className":37767,"style":1657},[529],[47,37769],{},[47,37771,1187],{"className":37772},[512],[47,37774,37776,37779],{"className":37775},[508],[47,37777,285],{"className":37778},[508,581],[47,37780,37782],{"className":37781},[585],[47,37783,37785,37816],{"className":37784},[520,521],[47,37786,37788,37813],{"className":37787},[525],[47,37789,37791,37802],{"className":37790,"style":36767},[529],[47,37792,37793,37796],{"style":17673},[47,37794],{"className":37795,"style":929},[537],[47,37797,37799],{"className":37798},[542,543,544,545],[47,37800,724],{"className":37801},[508,581,545],[47,37803,37804,37807],{"style":16891},[47,37805],{"className":37806,"style":929},[537],[47,37808,37810],{"className":37809},[542,543,544,545],[47,37811,1166],{"className":37812},[508,581,545],[47,37814,616],{"className":37815},[615],[47,37817,37819],{"className":37818},[525],[47,37820,37822],{"className":37821,"style":17703},[529],[47,37823],{},[47,37825,13303],{"className":37826},[508],[47,37828,37830,37833],{"className":37829},[508],[47,37831,13164],{"className":37832},[508,581],[47,37834,37836],{"className":37835},[585],[47,37837,37839,37870],{"className":37838},[520,521],[47,37840,37842,37867],{"className":37841},[525],[47,37843,37845,37856],{"className":37844,"style":36767},[529],[47,37846,37847,37850],{"style":17673},[47,37848],{"className":37849,"style":929},[537],[47,37851,37853],{"className":37852},[542,543,544,545],[47,37854,724],{"className":37855},[508,581,545],[47,37857,37858,37861],{"style":16891},[47,37859],{"className":37860,"style":929},[537],[47,37862,37864],{"className":37863},[542,543,544,545],[47,37865,1166],{"className":37866},[508,581,545],[47,37868,616],{"className":37869},[615],[47,37871,37873],{"className":37872},[525],[47,37874,37876],{"className":37875,"style":17703},[529],[47,37877],{},[47,37879,1201],{"className":37880},[702],[47,37882,37883,37886],{"style":663},[47,37884],{"className":37885,"style":538},[537],[47,37887],{"className":37888,"style":671},[670],[47,37890,37891,37894],{"style":16247},[47,37892],{"className":37893,"style":538},[537],[47,37895,37897,37937,37940,37991,37994,38045],{"className":37896},[508],[47,37898,37900,37903],{"className":37899},[508],[47,37901,5974],{"className":37902,"style":3555},[508,581],[47,37904,37906],{"className":37905},[585],[47,37907,37909,37929],{"className":37908},[520,521],[47,37910,37912,37926],{"className":37911},[525],[47,37913,37915],{"className":37914,"style":1635},[529],[47,37916,37917,37920],{"style":7830},[47,37918],{"className":37919,"style":929},[537],[47,37921,37923],{"className":37922},[542,543,544,545],[47,37924,14413],{"className":37925,"style":3580},[508,581,545],[47,37927,616],{"className":37928},[615],[47,37930,37932],{"className":37931},[525],[47,37933,37935],{"className":37934,"style":1657},[529],[47,37936],{},[47,37938,1187],{"className":37939},[512],[47,37941,37943,37946],{"className":37942},[508],[47,37944,285],{"className":37945},[508,581],[47,37947,37949],{"className":37948},[585],[47,37950,37952,37983],{"className":37951},[520,521],[47,37953,37955,37980],{"className":37954},[525],[47,37956,37958,37969],{"className":37957,"style":19211},[529],[47,37959,37960,37963],{"style":17673},[47,37961],{"className":37962,"style":929},[537],[47,37964,37966],{"className":37965},[542,543,544,545],[47,37967,724],{"className":37968},[508,581,545],[47,37970,37971,37974],{"style":925},[47,37972],{"className":37973,"style":929},[537],[47,37975,37977],{"className":37976},[542,543,544,545],[47,37978,1166],{"className":37979},[508,581,545],[47,37981,616],{"className":37982},[615],[47,37984,37986],{"className":37985},[525],[47,37987,37989],{"className":37988,"style":17703},[529],[47,37990],{},[47,37992,13303],{"className":37993},[508],[47,37995,37997,38000],{"className":37996},[508],[47,37998,13164],{"className":37999},[508,581],[47,38001,38003],{"className":38002},[585],[47,38004,38006,38037],{"className":38005},[520,521],[47,38007,38009,38034],{"className":38008},[525],[47,38010,38012,38023],{"className":38011,"style":19211},[529],[47,38013,38014,38017],{"style":17673},[47,38015],{"className":38016,"style":929},[537],[47,38018,38020],{"className":38019},[542,543,544,545],[47,38021,724],{"className":38022},[508,581,545],[47,38024,38025,38028],{"style":925},[47,38026],{"className":38027,"style":929},[537],[47,38029,38031],{"className":38030},[542,543,544,545],[47,38032,1166],{"className":38033},[508,581,545],[47,38035,616],{"className":38036},[615],[47,38038,38040],{"className":38039},[525],[47,38041,38043],{"className":38042,"style":17703},[529],[47,38044],{},[47,38046,1201],{"className":38047},[702],[47,38049,616],{"className":38050},[615],[47,38052,38054],{"className":38053},[525],[47,38055,38057],{"className":38056,"style":16312},[529],[47,38058],{},[47,38060],{"className":38061},[702,513],[47,38063,38065,38068,38420],{"className":38064},[508],[47,38066],{"className":38067},[512,513],[47,38069,38071],{"className":38070},[467],[47,38072,38074,38412],{"className":38073},[520,521],[47,38075,38077,38409],{"className":38076},[525],[47,38078,38080,38239,38247],{"className":38079,"style":36665},[529],[47,38081,38082,38085],{"style":16181},[47,38083],{"className":38084,"style":538},[537],[47,38086,38088,38128,38131,38182,38185,38236],{"className":38087},[508],[47,38089,38091,38094],{"className":38090},[508],[47,38092,5974],{"className":38093,"style":3555},[508,581],[47,38095,38097],{"className":38096},[585],[47,38098,38100,38120],{"className":38099},[520,521],[47,38101,38103,38117],{"className":38102},[525],[47,38104,38106],{"className":38105,"style":1635},[529],[47,38107,38108,38111],{"style":7830},[47,38109],{"className":38110,"style":929},[537],[47,38112,38114],{"className":38113},[542,543,544,545],[47,38115,14413],{"className":38116,"style":3580},[508,581,545],[47,38118,616],{"className":38119},[615],[47,38121,38123],{"className":38122},[525],[47,38124,38126],{"className":38125,"style":1657},[529],[47,38127],{},[47,38129,1187],{"className":38130},[512],[47,38132,38134,38137],{"className":38133},[508],[47,38135,285],{"className":38136},[508,581],[47,38138,38140],{"className":38139},[585],[47,38141,38143,38174],{"className":38142},[520,521],[47,38144,38146,38171],{"className":38145},[525],[47,38147,38149,38160],{"className":38148,"style":36767},[529],[47,38150,38151,38154],{"style":17673},[47,38152],{"className":38153,"style":929},[537],[47,38155,38157],{"className":38156},[542,543,544,545],[47,38158,724],{"className":38159},[508,581,545],[47,38161,38162,38165],{"style":16891},[47,38163],{"className":38164,"style":929},[537],[47,38166,38168],{"className":38167},[542,543,544,545],[47,38169,1166],{"className":38170},[508,581,545],[47,38172,616],{"className":38173},[615],[47,38175,38177],{"className":38176},[525],[47,38178,38180],{"className":38179,"style":17703},[529],[47,38181],{},[47,38183,13303],{"className":38184},[508],[47,38186,38188,38191],{"className":38187},[508],[47,38189,13164],{"className":38190},[508,581],[47,38192,38194],{"className":38193},[585],[47,38195,38197,38228],{"className":38196},[520,521],[47,38198,38200,38225],{"className":38199},[525],[47,38201,38203,38214],{"className":38202,"style":36767},[529],[47,38204,38205,38208],{"style":17673},[47,38206],{"className":38207,"style":929},[537],[47,38209,38211],{"className":38210},[542,543,544,545],[47,38212,724],{"className":38213},[508,581,545],[47,38215,38216,38219],{"style":16891},[47,38217],{"className":38218,"style":929},[537],[47,38220,38222],{"className":38221},[542,543,544,545],[47,38223,1166],{"className":38224},[508,581,545],[47,38226,616],{"className":38227},[615],[47,38229,38231],{"className":38230},[525],[47,38232,38234],{"className":38233,"style":17703},[529],[47,38235],{},[47,38237,1201],{"className":38238},[702],[47,38240,38241,38244],{"style":663},[47,38242],{"className":38243,"style":538},[537],[47,38245],{"className":38246,"style":671},[670],[47,38248,38249,38252],{"style":16247},[47,38250],{"className":38251,"style":538},[537],[47,38253,38255,38258,38298,38301,38352,38355,38406],{"className":38254},[508],[47,38256,14742],{"className":38257},[508],[47,38259,38261,38264],{"className":38260},[508],[47,38262,5974],{"className":38263,"style":3555},[508,581],[47,38265,38267],{"className":38266},[585],[47,38268,38270,38290],{"className":38269},[520,521],[47,38271,38273,38287],{"className":38272},[525],[47,38274,38276],{"className":38275,"style":1635},[529],[47,38277,38278,38281],{"style":7830},[47,38279],{"className":38280,"style":929},[537],[47,38282,38284],{"className":38283},[542,543,544,545],[47,38285,14413],{"className":38286,"style":3580},[508,581,545],[47,38288,616],{"className":38289},[615],[47,38291,38293],{"className":38292},[525],[47,38294,38296],{"className":38295,"style":1657},[529],[47,38297],{},[47,38299,1187],{"className":38300},[512],[47,38302,38304,38307],{"className":38303},[508],[47,38305,285],{"className":38306},[508,581],[47,38308,38310],{"className":38309},[585],[47,38311,38313,38344],{"className":38312},[520,521],[47,38314,38316,38341],{"className":38315},[525],[47,38317,38319,38330],{"className":38318,"style":19211},[529],[47,38320,38321,38324],{"style":17673},[47,38322],{"className":38323,"style":929},[537],[47,38325,38327],{"className":38326},[542,543,544,545],[47,38328,724],{"className":38329},[508,581,545],[47,38331,38332,38335],{"style":925},[47,38333],{"className":38334,"style":929},[537],[47,38336,38338],{"className":38337},[542,543,544,545],[47,38339,1166],{"className":38340},[508,581,545],[47,38342,616],{"className":38343},[615],[47,38345,38347],{"className":38346},[525],[47,38348,38350],{"className":38349,"style":17703},[529],[47,38351],{},[47,38353,13303],{"className":38354},[508],[47,38356,38358,38361],{"className":38357},[508],[47,38359,13164],{"className":38360},[508,581],[47,38362,38364],{"className":38363},[585],[47,38365,38367,38398],{"className":38366},[520,521],[47,38368,38370,38395],{"className":38369},[525],[47,38371,38373,38384],{"className":38372,"style":19211},[529],[47,38374,38375,38378],{"style":17673},[47,38376],{"className":38377,"style":929},[537],[47,38379,38381],{"className":38380},[542,543,544,545],[47,38382,724],{"className":38383},[508,581,545],[47,38385,38386,38389],{"style":925},[47,38387],{"className":38388,"style":929},[537],[47,38390,38392],{"className":38391},[542,543,544,545],[47,38393,1166],{"className":38394},[508,581,545],[47,38396,616],{"className":38397},[615],[47,38399,38401],{"className":38400},[525],[47,38402,38404],{"className":38403,"style":17703},[529],[47,38405],{},[47,38407,1201],{"className":38408},[702],[47,38410,616],{"className":38411},[615],[47,38413,38415],{"className":38414},[525],[47,38416,38418],{"className":38417,"style":16312},[529],[47,38419],{},[47,38421],{"className":38422},[702,513],[47,38424,38425,38428],{"style":35556},[47,38426],{"className":38427,"style":15532},[537],[47,38429,38431,38434,38437,38440,38443,38505,38508,38575,38578,38682,38685,38777,38780,38831,38834,38837,38888,38891],{"className":38430},[508],[47,38432],{"className":38433},[508],[47,38435],{"className":38436,"style":889},[888],[47,38438,82],{"className":38439},[893],[47,38441],{"className":38442,"style":889},[888],[47,38444,38446,38449,38502],{"className":38445},[508],[47,38447],{"className":38448},[512,513],[47,38450,38452],{"className":38451},[467],[47,38453,38455,38494],{"className":38454},[520,521],[47,38456,38458,38491],{"className":38457},[525],[47,38459,38461,38472,38480],{"className":38460,"style":16641},[529],[47,38462,38463,38466],{"style":16181},[47,38464],{"className":38465,"style":538},[537],[47,38467,38469],{"className":38468},[508],[47,38470,7692],{"className":38471,"style":7767},[508,581],[47,38473,38474,38477],{"style":663},[47,38475],{"className":38476,"style":538},[537],[47,38478],{"className":38479,"style":671},[670],[47,38481,38482,38485],{"style":16247},[47,38483],{"className":38484,"style":538},[537],[47,38486,38488],{"className":38487},[508],[47,38489,472],{"className":38490},[508],[47,38492,616],{"className":38493},[615],[47,38495,38497],{"className":38496},[525],[47,38498,38500],{"className":38499,"style":16681},[529],[47,38501],{},[47,38503],{"className":38504},[702,513],[47,38506],{"className":38507,"style":1303},[888],[47,38509,38511],{"className":38510},[1338,14600],[47,38512,38514,38567],{"className":38513},[520,521],[47,38515,38517,38564],{"className":38516},[525],[47,38518,38520,38540,38550],{"className":38519,"style":16702},[529],[47,38521,38522,38525],{"style":16705},[47,38523],{"className":38524,"style":15794},[537],[47,38526,38528],{"className":38527},[542,543,544,545],[47,38529,38531,38534,38537],{"className":38530},[508,545],[47,38532,724],{"className":38533},[508,581,545],[47,38535,82],{"className":38536},[893,545],[47,38538,472],{"className":38539},[508,545],[47,38541,38542,38545],{"style":15803},[47,38543],{"className":38544,"style":15794},[537],[47,38546,38547],{},[47,38548,2213],{"className":38549},[1338,2297,15812],[47,38551,38552,38555],{"style":16736},[47,38553],{"className":38554,"style":15794},[537],[47,38556,38558],{"className":38557},[542,543,544,545],[47,38559,38561],{"className":38560},[508,545],[47,38562,7692],{"className":38563,"style":7767},[508,581,545],[47,38565,616],{"className":38566},[615],[47,38568,38570],{"className":38569},[525],[47,38571,38573],{"className":38572,"style":16755},[529],[47,38574],{},[47,38576],{"className":38577,"style":1303},[888],[47,38579,38581],{"className":38580},[1338,14600],[47,38582,38584,38674],{"className":38583},[520,521],[47,38585,38587,38671],{"className":38586},[525],[47,38588,38590,38610,38620],{"className":38589,"style":17511},[529],[47,38591,38592,38595],{"style":16705},[47,38593],{"className":38594,"style":15794},[537],[47,38596,38598],{"className":38597},[542,543,544,545],[47,38599,38601,38604,38607],{"className":38600},[508,545],[47,38602,1166],{"className":38603},[508,581,545],[47,38605,82],{"className":38606},[893,545],[47,38608,472],{"className":38609},[508,545],[47,38611,38612,38615],{"style":15803},[47,38613],{"className":38614,"style":15794},[537],[47,38616,38617],{},[47,38618,2213],{"className":38619},[1338,2297,15812],[47,38621,38622,38625],{"style":17544},[47,38623],{"className":38624,"style":15794},[537],[47,38626,38628],{"className":38627},[542,543,544,545],[47,38629,38631],{"className":38630},[508,545],[47,38632,38634,38637],{"className":38633},[508,545],[47,38635,13604],{"className":38636,"style":3555},[508,581,545],[47,38638,38640],{"className":38639},[585],[47,38641,38643,38663],{"className":38642},[520,521],[47,38644,38646,38660],{"className":38645},[525],[47,38647,38649],{"className":38648,"style":4672},[529],[47,38650,38651,38654],{"style":17574},[47,38652],{"className":38653,"style":602},[537],[47,38655,38657],{"className":38656},[542,606,607,545],[47,38658,724],{"className":38659},[508,581,545],[47,38661,616],{"className":38662},[615],[47,38664,38666],{"className":38665},[525],[47,38667,38669],{"className":38668,"style":4694},[529],[47,38670],{},[47,38672,616],{"className":38673},[615],[47,38675,38677],{"className":38676},[525],[47,38678,38680],{"className":38679,"style":16755},[529],[47,38681],{},[47,38683],{"className":38684,"style":1303},[888],[47,38686,38688,38691],{"className":38687},[508],[47,38689,1163],{"className":38690},[508,581],[47,38692,38694],{"className":38693},[585],[47,38695,38697,38769],{"className":38696},[520,521],[47,38698,38700,38766],{"className":38699},[525],[47,38701,38703,38746],{"className":38702,"style":25158},[529],[47,38704,38705,38708],{"style":17673},[47,38706],{"className":38707,"style":929},[537],[47,38709,38711],{"className":38710},[542,543,544,545],[47,38712,38714],{"className":38713},[508,545],[47,38715,38717,38720],{"className":38716},[508,545],[47,38718,14413],{"className":38719,"style":3580},[508,581,545],[47,38721,38723],{"className":38722},[585],[47,38724,38726],{"className":38725},[520],[47,38727,38729],{"className":38728},[525],[47,38730,38732],{"className":38731,"style":19506},[529],[47,38733,38734,38737],{"style":19509},[47,38735],{"className":38736,"style":602},[537],[47,38738,38740],{"className":38739},[542,606,607,545],[47,38741,38743],{"className":38742},[508,545],[47,38744,19346],{"className":38745},[508,545],[47,38747,38748,38751],{"style":16788},[47,38749],{"className":38750,"style":929},[537],[47,38752,38754],{"className":38753},[542,543,544,545],[47,38755,38757,38760,38763],{"className":38756},[508,545],[47,38758,27872],{"className":38759},[508,581,545],[47,38761,1163],{"className":38762},[508,581,545],[47,38764,5977],{"className":38765,"style":6420},[508,581,545],[47,38767,616],{"className":38768},[615],[47,38770,38772],{"className":38771},[525],[47,38773,38775],{"className":38774,"style":17703},[529],[47,38776],{},[47,38778,1187],{"className":38779},[512],[47,38781,38783,38786],{"className":38782},[508],[47,38784,13164],{"className":38785},[508,581],[47,38787,38789],{"className":38788},[585],[47,38790,38792,38823],{"className":38791},[520,521],[47,38793,38795,38820],{"className":38794},[525],[47,38796,38798,38809],{"className":38797,"style":17670},[529],[47,38799,38800,38803],{"style":17673},[47,38801],{"className":38802,"style":929},[537],[47,38804,38806],{"className":38805},[542,543,544,545],[47,38807,724],{"className":38808},[508,581,545],[47,38810,38811,38814],{"style":16788},[47,38812],{"className":38813,"style":929},[537],[47,38815,38817],{"className":38816},[542,543,544,545],[47,38818,1166],{"className":38819},[508,581,545],[47,38821,616],{"className":38822},[615],[47,38824,38826],{"className":38825},[525],[47,38827,38829],{"className":38828,"style":17703},[529],[47,38830],{},[47,38832,1192],{"className":38833},[1299],[47,38835],{"className":38836,"style":1303},[888],[47,38838,38840,38843],{"className":38839},[508],[47,38841,285],{"className":38842},[508,581],[47,38844,38846],{"className":38845},[585],[47,38847,38849,38880],{"className":38848},[520,521],[47,38850,38852,38877],{"className":38851},[525],[47,38853,38855,38866],{"className":38854,"style":17670},[529],[47,38856,38857,38860],{"style":17673},[47,38858],{"className":38859,"style":929},[537],[47,38861,38863],{"className":38862},[542,543,544,545],[47,38864,724],{"className":38865},[508,581,545],[47,38867,38868,38871],{"style":16788},[47,38869],{"className":38870,"style":929},[537],[47,38872,38874],{"className":38873},[542,543,544,545],[47,38875,1166],{"className":38876},[508,581,545],[47,38878,616],{"className":38879},[615],[47,38881,38883],{"className":38882},[525],[47,38884,38886],{"className":38885,"style":17703},[529],[47,38887],{},[47,38889,1201],{"className":38890},[702],[47,38892,38894,38897,39281],{"className":38893},[508],[47,38895],{"className":38896},[512,513],[47,38898,38900],{"className":38899},[467],[47,38901,38903,39273],{"className":38902},[520,521],[47,38904,38906,39270],{"className":38905},[525],[47,38907,38909,39100,39108],{"className":38908,"style":36665},[529],[47,38910,38911,38914],{"style":16181},[47,38912],{"className":38913,"style":538},[537],[47,38915,38917,38989,38992,39043,39046,39097],{"className":38916},[508],[47,38918,38920,38923],{"className":38919},[508],[47,38921,5974],{"className":38922,"style":3555},[508,581],[47,38924,38926],{"className":38925},[585],[47,38927,38929,38981],{"className":38928},[520,521],[47,38930,38932,38978],{"className":38931},[525],[47,38933,38935],{"className":38934,"style":1635},[529],[47,38936,38937,38940],{"style":7830},[47,38938],{"className":38939,"style":929},[537],[47,38941,38943],{"className":38942},[542,543,544,545],[47,38944,38946],{"className":38945},[508,545],[47,38947,38949,38952],{"className":38948},[508,545],[47,38950,14413],{"className":38951,"style":3580},[508,581,545],[47,38953,38955],{"className":38954},[585],[47,38956,38958],{"className":38957},[520],[47,38959,38961],{"className":38960},[525],[47,38962,38964],{"className":38963,"style":19506},[529],[47,38965,38966,38969],{"style":19509},[47,38967],{"className":38968,"style":602},[537],[47,38970,38972],{"className":38971},[542,606,607,545],[47,38973,38975],{"className":38974},[508,545],[47,38976,19346],{"className":38977},[508,545],[47,38979,616],{"className":38980},[615],[47,38982,38984],{"className":38983},[525],[47,38985,38987],{"className":38986,"style":1657},[529],[47,38988],{},[47,38990,1187],{"className":38991},[512],[47,38993,38995,38998],{"className":38994},[508],[47,38996,285],{"className":38997},[508,581],[47,38999,39001],{"className":39000},[585],[47,39002,39004,39035],{"className":39003},[520,521],[47,39005,39007,39032],{"className":39006},[525],[47,39008,39010,39021],{"className":39009,"style":36767},[529],[47,39011,39012,39015],{"style":17673},[47,39013],{"className":39014,"style":929},[537],[47,39016,39018],{"className":39017},[542,543,544,545],[47,39019,724],{"className":39020},[508,581,545],[47,39022,39023,39026],{"style":16891},[47,39024],{"className":39025,"style":929},[537],[47,39027,39029],{"className":39028},[542,543,544,545],[47,39030,1166],{"className":39031},[508,581,545],[47,39033,616],{"className":39034},[615],[47,39036,39038],{"className":39037},[525],[47,39039,39041],{"className":39040,"style":17703},[529],[47,39042],{},[47,39044,13303],{"className":39045},[508],[47,39047,39049,39052],{"className":39048},[508],[47,39050,13164],{"className":39051},[508,581],[47,39053,39055],{"className":39054},[585],[47,39056,39058,39089],{"className":39057},[520,521],[47,39059,39061,39086],{"className":39060},[525],[47,39062,39064,39075],{"className":39063,"style":36767},[529],[47,39065,39066,39069],{"style":17673},[47,39067],{"className":39068,"style":929},[537],[47,39070,39072],{"className":39071},[542,543,544,545],[47,39073,724],{"className":39074},[508,581,545],[47,39076,39077,39080],{"style":16891},[47,39078],{"className":39079,"style":929},[537],[47,39081,39083],{"className":39082},[542,543,544,545],[47,39084,1166],{"className":39085},[508,581,545],[47,39087,616],{"className":39088},[615],[47,39090,39092],{"className":39091},[525],[47,39093,39095],{"className":39094,"style":17703},[529],[47,39096],{},[47,39098,1201],{"className":39099},[702],[47,39101,39102,39105],{"style":663},[47,39103],{"className":39104,"style":538},[537],[47,39106],{"className":39107,"style":671},[670],[47,39109,39110,39113],{"style":16247},[47,39111],{"className":39112,"style":538},[537],[47,39114,39116,39119,39159,39162,39213,39216,39267],{"className":39115},[508],[47,39117,14742],{"className":39118},[508],[47,39120,39122,39125],{"className":39121},[508],[47,39123,5974],{"className":39124,"style":3555},[508,581],[47,39126,39128],{"className":39127},[585],[47,39129,39131,39151],{"className":39130},[520,521],[47,39132,39134,39148],{"className":39133},[525],[47,39135,39137],{"className":39136,"style":1635},[529],[47,39138,39139,39142],{"style":7830},[47,39140],{"className":39141,"style":929},[537],[47,39143,39145],{"className":39144},[542,543,544,545],[47,39146,14413],{"className":39147,"style":3580},[508,581,545],[47,39149,616],{"className":39150},[615],[47,39152,39154],{"className":39153},[525],[47,39155,39157],{"className":39156,"style":1657},[529],[47,39158],{},[47,39160,1187],{"className":39161},[512],[47,39163,39165,39168],{"className":39164},[508],[47,39166,285],{"className":39167},[508,581],[47,39169,39171],{"className":39170},[585],[47,39172,39174,39205],{"className":39173},[520,521],[47,39175,39177,39202],{"className":39176},[525],[47,39178,39180,39191],{"className":39179,"style":19211},[529],[47,39181,39182,39185],{"style":17673},[47,39183],{"className":39184,"style":929},[537],[47,39186,39188],{"className":39187},[542,543,544,545],[47,39189,724],{"className":39190},[508,581,545],[47,39192,39193,39196],{"style":925},[47,39194],{"className":39195,"style":929},[537],[47,39197,39199],{"className":39198},[542,543,544,545],[47,39200,1166],{"className":39201},[508,581,545],[47,39203,616],{"className":39204},[615],[47,39206,39208],{"className":39207},[525],[47,39209,39211],{"className":39210,"style":17703},[529],[47,39212],{},[47,39214,13303],{"className":39215},[508],[47,39217,39219,39222],{"className":39218},[508],[47,39220,13164],{"className":39221},[508,581],[47,39223,39225],{"className":39224},[585],[47,39226,39228,39259],{"className":39227},[520,521],[47,39229,39231,39256],{"className":39230},[525],[47,39232,39234,39245],{"className":39233,"style":19211},[529],[47,39235,39236,39239],{"style":17673},[47,39237],{"className":39238,"style":929},[537],[47,39240,39242],{"className":39241},[542,543,544,545],[47,39243,724],{"className":39244},[508,581,545],[47,39246,39247,39250],{"style":925},[47,39248],{"className":39249,"style":929},[537],[47,39251,39253],{"className":39252},[542,543,544,545],[47,39254,1166],{"className":39255},[508,581,545],[47,39257,616],{"className":39258},[615],[47,39260,39262],{"className":39261},[525],[47,39263,39265],{"className":39264,"style":17703},[529],[47,39266],{},[47,39268,1201],{"className":39269},[702],[47,39271,616],{"className":39272},[615],[47,39274,39276],{"className":39275},[525],[47,39277,39279],{"className":39278,"style":16312},[529],[47,39280],{},[47,39282],{"className":39283},[702,513],[47,39285,616],{"className":39286},[615],[47,39288,39290],{"className":39289},[525],[47,39291,39293],{"className":39292,"style":35572},[529],[47,39294],{},[14,39296,39297],{},"对应地，极大化目标时可用如下损失（最小化时取负号）：",[47,39299,39301],{"className":39300},[14712],[47,39302,39304,39472],{"className":39303},[451],[47,39305,39307],{"className":39306},[455],[457,39308,39309],{"xmlns":459,"display":9848},[461,39310,39311,39469],{},[464,39312,39313,39315,39317,39319,39321,39323,39325,39331,39345,39363,39381,39383,39391,39393,39401,39403],{},[480,39314,8186],{},[480,39316,1181],{},[480,39318,13164],{},[480,39320,13164],{},[850,39322,82],{},[850,39324,18727],{},[467,39326,39327,39329],{},[470,39328,472],{},[480,39330,7692],{},[15045,39332,39333,39335,39343],{},[850,39334,2213],{},[464,39336,39337,39339,39341],{},[480,39338,724],{},[850,39340,82],{},[470,39342,472],{},[480,39344,7692],{},[15045,39346,39347,39349,39357],{},[850,39348,2213],{},[464,39350,39351,39353,39355],{},[480,39352,1166],{},[850,39354,82],{},[470,39356,472],{},[477,39358,39359,39361],{},[480,39360,13604],{},[480,39362,724],{},[2209,39364,39365,39367,39373],{},[480,39366,1163],{},[854,39368,39369,39371],{},[480,39370,14413],{},[850,39372,19346],{"mathvariant":1206,"lspace":14728,"rspace":14728},[464,39374,39375,39377,39379],{},[480,39376,27872],{},[480,39378,1163],{},[480,39380,5977],{},[850,39382,1187],{"stretchy":1186},[2209,39384,39385,39387,39389],{},[480,39386,13164],{},[480,39388,724],{},[480,39390,1166],{},[850,39392,1192],{"separator":495},[2209,39394,39395,39397,39399],{},[480,39396,285],{},[480,39398,724],{},[480,39400,1166],{},[850,39402,1201],{"stretchy":1186},[467,39404,39405,39435],{},[464,39406,39407,39413,39415,39423,39425,39433],{},[477,39408,39409,39411],{},[480,39410,5974],{},[480,39412,14413],{},[850,39414,1187],{"stretchy":1186},[2209,39416,39417,39419,39421],{},[480,39418,285],{},[480,39420,724],{},[480,39422,1166],{},[480,39424,13303],{"mathvariant":1206},[2209,39426,39427,39429,39431],{},[480,39428,13164],{},[480,39430,724],{},[480,39432,1166],{},[850,39434,1201],{"stretchy":1186},[464,39436,39437,39447,39449,39457,39459,39467],{},[477,39438,39439,39441],{},[480,39440,5974],{},[854,39442,39443,39445],{},[480,39444,14413],{},[850,39446,19346],{"mathvariant":1206,"lspace":14728,"rspace":14728},[850,39448,1187],{"stretchy":1186},[2209,39450,39451,39453,39455],{},[480,39452,285],{},[480,39454,724],{},[480,39456,1166],{},[480,39458,13303],{"mathvariant":1206},[2209,39460,39461,39463,39465],{},[480,39462,13164],{},[480,39464,724],{},[480,39466,1166],{},[850,39468,1201],{"stretchy":1186},[487,39470,39471],{"encoding":489},"Loss = -\\frac{1}{N} \\sum_{n=1}^{N} \\sum_{t=1}^{T_n} A_{\\theta'}^{GAE}(s_n^t, a_n^t) \\frac{P_\\theta(a_n^t | s_n^t)}{P_{\\theta'}(a_n^t | s_n^t)}",[47,39473,39475,39496],{"className":39474,"ariaHidden":495},[494],[47,39476,39478,39481,39484,39487,39490,39493],{"className":39477},[499],[47,39479],{"className":39480,"style":14365},[503],[47,39482,8186],{"className":39483},[508,581],[47,39485,18828],{"className":39486},[508,581],[47,39488],{"className":39489,"style":889},[888],[47,39491,82],{"className":39492},[893],[47,39494],{"className":39495,"style":889},[888],[47,39497,39499,39502,39505,39567,39570,39637,39640,39744,39747,39839,39842,39893,39896,39899,39950,39953],{"className":39498},[499],[47,39500],{"className":39501,"style":31535},[503],[47,39503,18727],{"className":39504},[508],[47,39506,39508,39511,39564],{"className":39507},[508],[47,39509],{"className":39510},[512,513],[47,39512,39514],{"className":39513},[467],[47,39515,39517,39556],{"className":39516},[520,521],[47,39518,39520,39553],{"className":39519},[525],[47,39521,39523,39534,39542],{"className":39522,"style":16641},[529],[47,39524,39525,39528],{"style":16181},[47,39526],{"className":39527,"style":538},[537],[47,39529,39531],{"className":39530},[508],[47,39532,7692],{"className":39533,"style":7767},[508,581],[47,39535,39536,39539],{"style":663},[47,39537],{"className":39538,"style":538},[537],[47,39540],{"className":39541,"style":671},[670],[47,39543,39544,39547],{"style":16247},[47,39545],{"className":39546,"style":538},[537],[47,39548,39550],{"className":39549},[508],[47,39551,472],{"className":39552},[508],[47,39554,616],{"className":39555},[615],[47,39557,39559],{"className":39558},[525],[47,39560,39562],{"className":39561,"style":16681},[529],[47,39563],{},[47,39565],{"className":39566},[702,513],[47,39568],{"className":39569,"style":1303},[888],[47,39571,39573],{"className":39572},[1338,14600],[47,39574,39576,39629],{"className":39575},[520,521],[47,39577,39579,39626],{"className":39578},[525],[47,39580,39582,39602,39612],{"className":39581,"style":16702},[529],[47,39583,39584,39587],{"style":16705},[47,39585],{"className":39586,"style":15794},[537],[47,39588,39590],{"className":39589},[542,543,544,545],[47,39591,39593,39596,39599],{"className":39592},[508,545],[47,39594,724],{"className":39595},[508,581,545],[47,39597,82],{"className":39598},[893,545],[47,39600,472],{"className":39601},[508,545],[47,39603,39604,39607],{"style":15803},[47,39605],{"className":39606,"style":15794},[537],[47,39608,39609],{},[47,39610,2213],{"className":39611},[1338,2297,15812],[47,39613,39614,39617],{"style":16736},[47,39615],{"className":39616,"style":15794},[537],[47,39618,39620],{"className":39619},[542,543,544,545],[47,39621,39623],{"className":39622},[508,545],[47,39624,7692],{"className":39625,"style":7767},[508,581,545],[47,39627,616],{"className":39628},[615],[47,39630,39632],{"className":39631},[525],[47,39633,39635],{"className":39634,"style":16755},[529],[47,39636],{},[47,39638],{"className":39639,"style":1303},[888],[47,39641,39643],{"className":39642},[1338,14600],[47,39644,39646,39736],{"className":39645},[520,521],[47,39647,39649,39733],{"className":39648},[525],[47,39650,39652,39672,39682],{"className":39651,"style":17511},[529],[47,39653,39654,39657],{"style":16705},[47,39655],{"className":39656,"style":15794},[537],[47,39658,39660],{"className":39659},[542,543,544,545],[47,39661,39663,39666,39669],{"className":39662},[508,545],[47,39664,1166],{"className":39665},[508,581,545],[47,39667,82],{"className":39668},[893,545],[47,39670,472],{"className":39671},[508,545],[47,39673,39674,39677],{"style":15803},[47,39675],{"className":39676,"style":15794},[537],[47,39678,39679],{},[47,39680,2213],{"className":39681},[1338,2297,15812],[47,39683,39684,39687],{"style":17544},[47,39685],{"className":39686,"style":15794},[537],[47,39688,39690],{"className":39689},[542,543,544,545],[47,39691,39693],{"className":39692},[508,545],[47,39694,39696,39699],{"className":39695},[508,545],[47,39697,13604],{"className":39698,"style":3555},[508,581,545],[47,39700,39702],{"className":39701},[585],[47,39703,39705,39725],{"className":39704},[520,521],[47,39706,39708,39722],{"className":39707},[525],[47,39709,39711],{"className":39710,"style":4672},[529],[47,39712,39713,39716],{"style":17574},[47,39714],{"className":39715,"style":602},[537],[47,39717,39719],{"className":39718},[542,606,607,545],[47,39720,724],{"className":39721},[508,581,545],[47,39723,616],{"className":39724},[615],[47,39726,39728],{"className":39727},[525],[47,39729,39731],{"className":39730,"style":4694},[529],[47,39732],{},[47,39734,616],{"className":39735},[615],[47,39737,39739],{"className":39738},[525],[47,39740,39742],{"className":39741,"style":16755},[529],[47,39743],{},[47,39745],{"className":39746,"style":1303},[888],[47,39748,39750,39753],{"className":39749},[508],[47,39751,1163],{"className":39752},[508,581],[47,39754,39756],{"className":39755},[585],[47,39757,39759,39831],{"className":39758},[520,521],[47,39760,39762,39828],{"className":39761},[525],[47,39763,39765,39808],{"className":39764,"style":25158},[529],[47,39766,39767,39770],{"style":17673},[47,39768],{"className":39769,"style":929},[537],[47,39771,39773],{"className":39772},[542,543,544,545],[47,39774,39776],{"className":39775},[508,545],[47,39777,39779,39782],{"className":39778},[508,545],[47,39780,14413],{"className":39781,"style":3580},[508,581,545],[47,39783,39785],{"className":39784},[585],[47,39786,39788],{"className":39787},[520],[47,39789,39791],{"className":39790},[525],[47,39792,39794],{"className":39793,"style":19506},[529],[47,39795,39796,39799],{"style":19509},[47,39797],{"className":39798,"style":602},[537],[47,39800,39802],{"className":39801},[542,606,607,545],[47,39803,39805],{"className":39804},[508,545],[47,39806,19346],{"className":39807},[508,545],[47,39809,39810,39813],{"style":16788},[47,39811],{"className":39812,"style":929},[537],[47,39814,39816],{"className":39815},[542,543,544,545],[47,39817,39819,39822,39825],{"className":39818},[508,545],[47,39820,27872],{"className":39821},[508,581,545],[47,39823,1163],{"className":39824},[508,581,545],[47,39826,5977],{"className":39827,"style":6420},[508,581,545],[47,39829,616],{"className":39830},[615],[47,39832,39834],{"className":39833},[525],[47,39835,39837],{"className":39836,"style":17703},[529],[47,39838],{},[47,39840,1187],{"className":39841},[512],[47,39843,39845,39848],{"className":39844},[508],[47,39846,13164],{"className":39847},[508,581],[47,39849,39851],{"className":39850},[585],[47,39852,39854,39885],{"className":39853},[520,521],[47,39855,39857,39882],{"className":39856},[525],[47,39858,39860,39871],{"className":39859,"style":17670},[529],[47,39861,39862,39865],{"style":17673},[47,39863],{"className":39864,"style":929},[537],[47,39866,39868],{"className":39867},[542,543,544,545],[47,39869,724],{"className":39870},[508,581,545],[47,39872,39873,39876],{"style":16788},[47,39874],{"className":39875,"style":929},[537],[47,39877,39879],{"className":39878},[542,543,544,545],[47,39880,1166],{"className":39881},[508,581,545],[47,39883,616],{"className":39884},[615],[47,39886,39888],{"className":39887},[525],[47,39889,39891],{"className":39890,"style":17703},[529],[47,39892],{},[47,39894,1192],{"className":39895},[1299],[47,39897],{"className":39898,"style":1303},[888],[47,39900,39902,39905],{"className":39901},[508],[47,39903,285],{"className":39904},[508,581],[47,39906,39908],{"className":39907},[585],[47,39909,39911,39942],{"className":39910},[520,521],[47,39912,39914,39939],{"className":39913},[525],[47,39915,39917,39928],{"className":39916,"style":17670},[529],[47,39918,39919,39922],{"style":17673},[47,39920],{"className":39921,"style":929},[537],[47,39923,39925],{"className":39924},[542,543,544,545],[47,39926,724],{"className":39927},[508,581,545],[47,39929,39930,39933],{"style":16788},[47,39931],{"className":39932,"style":929},[537],[47,39934,39936],{"className":39935},[542,543,544,545],[47,39937,1166],{"className":39938},[508,581,545],[47,39940,616],{"className":39941},[615],[47,39943,39945],{"className":39944},[525],[47,39946,39948],{"className":39947,"style":17703},[529],[47,39949],{},[47,39951,1201],{"className":39952},[702],[47,39954,39956,39959,40340],{"className":39955},[508],[47,39957],{"className":39958},[512,513],[47,39960,39962],{"className":39961},[467],[47,39963,39965,40332],{"className":39964},[520,521],[47,39966,39968,40329],{"className":39967},[525],[47,39969,39971,40162,40170],{"className":39970,"style":36665},[529],[47,39972,39973,39976],{"style":16181},[47,39974],{"className":39975,"style":538},[537],[47,39977,39979,40051,40054,40105,40108,40159],{"className":39978},[508],[47,39980,39982,39985],{"className":39981},[508],[47,39983,5974],{"className":39984,"style":3555},[508,581],[47,39986,39988],{"className":39987},[585],[47,39989,39991,40043],{"className":39990},[520,521],[47,39992,39994,40040],{"className":39993},[525],[47,39995,39997],{"className":39996,"style":1635},[529],[47,39998,39999,40002],{"style":7830},[47,40000],{"className":40001,"style":929},[537],[47,40003,40005],{"className":40004},[542,543,544,545],[47,40006,40008],{"className":40007},[508,545],[47,40009,40011,40014],{"className":40010},[508,545],[47,40012,14413],{"className":40013,"style":3580},[508,581,545],[47,40015,40017],{"className":40016},[585],[47,40018,40020],{"className":40019},[520],[47,40021,40023],{"className":40022},[525],[47,40024,40026],{"className":40025,"style":19506},[529],[47,40027,40028,40031],{"style":19509},[47,40029],{"className":40030,"style":602},[537],[47,40032,40034],{"className":40033},[542,606,607,545],[47,40035,40037],{"className":40036},[508,545],[47,40038,19346],{"className":40039},[508,545],[47,40041,616],{"className":40042},[615],[47,40044,40046],{"className":40045},[525],[47,40047,40049],{"className":40048,"style":1657},[529],[47,40050],{},[47,40052,1187],{"className":40053},[512],[47,40055,40057,40060],{"className":40056},[508],[47,40058,285],{"className":40059},[508,581],[47,40061,40063],{"className":40062},[585],[47,40064,40066,40097],{"className":40065},[520,521],[47,40067,40069,40094],{"className":40068},[525],[47,40070,40072,40083],{"className":40071,"style":36767},[529],[47,40073,40074,40077],{"style":17673},[47,40075],{"className":40076,"style":929},[537],[47,40078,40080],{"className":40079},[542,543,544,545],[47,40081,724],{"className":40082},[508,581,545],[47,40084,40085,40088],{"style":16891},[47,40086],{"className":40087,"style":929},[537],[47,40089,40091],{"className":40090},[542,543,544,545],[47,40092,1166],{"className":40093},[508,581,545],[47,40095,616],{"className":40096},[615],[47,40098,40100],{"className":40099},[525],[47,40101,40103],{"className":40102,"style":17703},[529],[47,40104],{},[47,40106,13303],{"className":40107},[508],[47,40109,40111,40114],{"className":40110},[508],[47,40112,13164],{"className":40113},[508,581],[47,40115,40117],{"className":40116},[585],[47,40118,40120,40151],{"className":40119},[520,521],[47,40121,40123,40148],{"className":40122},[525],[47,40124,40126,40137],{"className":40125,"style":36767},[529],[47,40127,40128,40131],{"style":17673},[47,40129],{"className":40130,"style":929},[537],[47,40132,40134],{"className":40133},[542,543,544,545],[47,40135,724],{"className":40136},[508,581,545],[47,40138,40139,40142],{"style":16891},[47,40140],{"className":40141,"style":929},[537],[47,40143,40145],{"className":40144},[542,543,544,545],[47,40146,1166],{"className":40147},[508,581,545],[47,40149,616],{"className":40150},[615],[47,40152,40154],{"className":40153},[525],[47,40155,40157],{"className":40156,"style":17703},[529],[47,40158],{},[47,40160,1201],{"className":40161},[702],[47,40163,40164,40167],{"style":663},[47,40165],{"className":40166,"style":538},[537],[47,40168],{"className":40169,"style":671},[670],[47,40171,40172,40175],{"style":16247},[47,40173],{"className":40174,"style":538},[537],[47,40176,40178,40218,40221,40272,40275,40326],{"className":40177},[508],[47,40179,40181,40184],{"className":40180},[508],[47,40182,5974],{"className":40183,"style":3555},[508,581],[47,40185,40187],{"className":40186},[585],[47,40188,40190,40210],{"className":40189},[520,521],[47,40191,40193,40207],{"className":40192},[525],[47,40194,40196],{"className":40195,"style":1635},[529],[47,40197,40198,40201],{"style":7830},[47,40199],{"className":40200,"style":929},[537],[47,40202,40204],{"className":40203},[542,543,544,545],[47,40205,14413],{"className":40206,"style":3580},[508,581,545],[47,40208,616],{"className":40209},[615],[47,40211,40213],{"className":40212},[525],[47,40214,40216],{"className":40215,"style":1657},[529],[47,40217],{},[47,40219,1187],{"className":40220},[512],[47,40222,40224,40227],{"className":40223},[508],[47,40225,285],{"className":40226},[508,581],[47,40228,40230],{"className":40229},[585],[47,40231,40233,40264],{"className":40232},[520,521],[47,40234,40236,40261],{"className":40235},[525],[47,40237,40239,40250],{"className":40238,"style":19211},[529],[47,40240,40241,40244],{"style":17673},[47,40242],{"className":40243,"style":929},[537],[47,40245,40247],{"className":40246},[542,543,544,545],[47,40248,724],{"className":40249},[508,581,545],[47,40251,40252,40255],{"style":925},[47,40253],{"className":40254,"style":929},[537],[47,40256,40258],{"className":40257},[542,543,544,545],[47,40259,1166],{"className":40260},[508,581,545],[47,40262,616],{"className":40263},[615],[47,40265,40267],{"className":40266},[525],[47,40268,40270],{"className":40269,"style":17703},[529],[47,40271],{},[47,40273,13303],{"className":40274},[508],[47,40276,40278,40281],{"className":40277},[508],[47,40279,13164],{"className":40280},[508,581],[47,40282,40284],{"className":40283},[585],[47,40285,40287,40318],{"className":40286},[520,521],[47,40288,40290,40315],{"className":40289},[525],[47,40291,40293,40304],{"className":40292,"style":19211},[529],[47,40294,40295,40298],{"style":17673},[47,40296],{"className":40297,"style":929},[537],[47,40299,40301],{"className":40300},[542,543,544,545],[47,40302,724],{"className":40303},[508,581,545],[47,40305,40306,40309],{"style":925},[47,40307],{"className":40308,"style":929},[537],[47,40310,40312],{"className":40311},[542,543,544,545],[47,40313,1166],{"className":40314},[508,581,545],[47,40316,616],{"className":40317},[615],[47,40319,40321],{"className":40320},[525],[47,40322,40324],{"className":40323,"style":17703},[529],[47,40325],{},[47,40327,1201],{"className":40328},[702],[47,40330,616],{"className":40331},[615],[47,40333,40335],{"className":40334},[525],[47,40336,40338],{"className":40337,"style":16312},[529],[47,40339],{},[47,40341],{"className":40342},[702,513],[14,40344,40345],{},"同时，你学习的目标模型不能和我们的差距过大，否则很难学到有用的经验和教训。",[40347,40348,40350],"h4",{"id":40349},"ppo-penalty在-surrogate-上加-kl-惩罚","PPO-Penalty：在 surrogate 上加 KL 惩罚",[14,40352,40353,40354,40386],{},"如何给我们训练的策略和参考的策略增加相差不能过大这个约束呢？那就是加上 KL 散度的约束。KL 散度就是衡量两个分布差异大小的指标。两个分布差异越小，KL散度越小，分布差异越大，KL散度越大。通过因子 ",[47,40355,40357,40372],{"className":40356},[451],[47,40358,40360],{"className":40359},[455],[457,40361,40362],{"xmlns":459},[461,40363,40364,40369],{},[464,40365,40366],{},[480,40367,40368],{},"β",[487,40370,40371],{"encoding":489},"\\beta",[47,40373,40375],{"className":40374,"ariaHidden":495},[494],[47,40376,40378,40382],{"className":40377},[499],[47,40379],{"className":40380,"style":40381},[503],"height:0.8889em;vertical-align:-0.1944em;",[47,40383,40368],{"className":40384,"style":40385},[508,581],"margin-right:0.05278em;"," 来调整 KL 散度影响的大小。",[47,40388,40390],{"className":40389},[14712],[47,40391,40393,40600],{"className":40392},[451],[47,40394,40396],{"className":40395},[455],[457,40397,40398],{"xmlns":459,"display":9848},[461,40399,40400,40597],{},[464,40401,40402,40404,40406,40408,40415,40417,40419,40425,40439,40457,40475,40477,40485,40487,40495,40497,40563,40565,40567,40569,40575,40577,40583,40585,40595],{},[480,40403,8186],{},[480,40405,1181],{},[480,40407,13164],{},[477,40409,40410,40412],{},[480,40411,13164],{},[3647,40413,40414],{},"ppo",[850,40416,82],{},[850,40418,18727],{},[467,40420,40421,40423],{},[470,40422,472],{},[480,40424,7692],{},[15045,40426,40427,40429,40437],{},[850,40428,2213],{},[464,40430,40431,40433,40435],{},[480,40432,724],{},[850,40434,82],{},[470,40436,472],{},[480,40438,7692],{},[15045,40440,40441,40443,40451],{},[850,40442,2213],{},[464,40444,40445,40447,40449],{},[480,40446,1166],{},[850,40448,82],{},[470,40450,472],{},[477,40452,40453,40455],{},[480,40454,13604],{},[480,40456,724],{},[2209,40458,40459,40461,40467],{},[480,40460,1163],{},[854,40462,40463,40465],{},[480,40464,14413],{},[850,40466,19346],{"mathvariant":1206,"lspace":14728,"rspace":14728},[464,40468,40469,40471,40473],{},[480,40470,27872],{},[480,40472,1163],{},[480,40474,5977],{},[850,40476,1187],{"stretchy":1186},[2209,40478,40479,40481,40483],{},[480,40480,13164],{},[480,40482,724],{},[480,40484,1166],{},[850,40486,1192],{"separator":495},[2209,40488,40489,40491,40493],{},[480,40490,285],{},[480,40492,724],{},[480,40494,1166],{},[850,40496,1201],{"stretchy":1186},[467,40498,40499,40529],{},[464,40500,40501,40507,40509,40517,40519,40527],{},[477,40502,40503,40505],{},[480,40504,5974],{},[480,40506,14413],{},[850,40508,1187],{"stretchy":1186},[2209,40510,40511,40513,40515],{},[480,40512,285],{},[480,40514,724],{},[480,40516,1166],{},[480,40518,13303],{"mathvariant":1206},[2209,40520,40521,40523,40525],{},[480,40522,13164],{},[480,40524,724],{},[480,40526,1166],{},[850,40528,1201],{"stretchy":1186},[464,40530,40531,40541,40543,40551,40553,40561],{},[477,40532,40533,40535],{},[480,40534,5974],{},[854,40536,40537,40539],{},[480,40538,14413],{},[850,40540,19346],{"mathvariant":1206,"lspace":14728,"rspace":14728},[850,40542,1187],{"stretchy":1186},[2209,40544,40545,40547,40549],{},[480,40546,285],{},[480,40548,724],{},[480,40550,1166],{},[480,40552,13303],{"mathvariant":1206},[2209,40554,40555,40557,40559],{},[480,40556,13164],{},[480,40558,724],{},[480,40560,1166],{},[850,40562,1201],{"stretchy":1186},[850,40564,6147],{},[480,40566,40368],{},[3647,40568,27957],{},[464,40570,40571,40573],{},[480,40572,961],{"mathvariant":1206},[480,40574,8186],{"mathvariant":1206},[850,40576,1187],{"stretchy":1186},[477,40578,40579,40581],{},[480,40580,5974],{},[480,40582,14413],{},[850,40584,1192],{"separator":495},[477,40586,40587,40589],{},[480,40588,5974],{},[854,40590,40591,40593],{},[480,40592,14413],{},[850,40594,19346],{"mathvariant":1206,"lspace":14728,"rspace":14728},[850,40596,1201],{"stretchy":1186},[487,40598,40599],{"encoding":489},"Loss_{\\text{ppo}} = -\\frac{1}{N} \\sum_{n=1}^{N} \\sum_{t=1}^{T_n} A_{\\theta'}^{GAE}(s_n^t, a_n^t) \\frac{P_{\\theta}(a_n^t|s_n^t)}{P_{\\theta'}(a_n^t|s_n^t)} + \\beta\\,\\mathrm{KL}(P_{\\theta}, P_{\\theta'})",[47,40601,40603,40673,41532],{"className":40602,"ariaHidden":495},[494],[47,40604,40606,40610,40613,40617,40664,40667,40670],{"className":40605},[499],[47,40607],{"className":40608,"style":40609},[503],"height:0.9694em;vertical-align:-0.2861em;",[47,40611,8186],{"className":40612},[508,581],[47,40614,40616],{"className":40615},[508,581],"os",[47,40618,40620,40623],{"className":40619},[508],[47,40621,13164],{"className":40622},[508,581],[47,40624,40626],{"className":40625},[585],[47,40627,40629,40655],{"className":40628},[520,521],[47,40630,40632,40652],{"className":40631},[525],[47,40633,40635],{"className":40634,"style":1709},[529],[47,40636,40637,40640],{"style":1638},[47,40638],{"className":40639,"style":929},[537],[47,40641,40643],{"className":40642},[542,543,544,545],[47,40644,40646],{"className":40645},[508,545],[47,40647,40649],{"className":40648},[508,12219,545],[47,40650,40414],{"className":40651},[508,545],[47,40653,616],{"className":40654},[615],[47,40656,40658],{"className":40657},[525],[47,40659,40662],{"className":40660,"style":40661},[529],"height:0.2861em;",[47,40663],{},[47,40665],{"className":40666,"style":889},[888],[47,40668,82],{"className":40669},[893],[47,40671],{"className":40672,"style":889},[888],[47,40674,40676,40679,40682,40744,40747,40814,40817,40921,40924,41016,41019,41070,41073,41076,41127,41130,41523,41526,41529],{"className":40675},[499],[47,40677],{"className":40678,"style":31535},[503],[47,40680,18727],{"className":40681},[508],[47,40683,40685,40688,40741],{"className":40684},[508],[47,40686],{"className":40687},[512,513],[47,40689,40691],{"className":40690},[467],[47,40692,40694,40733],{"className":40693},[520,521],[47,40695,40697,40730],{"className":40696},[525],[47,40698,40700,40711,40719],{"className":40699,"style":16641},[529],[47,40701,40702,40705],{"style":16181},[47,40703],{"className":40704,"style":538},[537],[47,40706,40708],{"className":40707},[508],[47,40709,7692],{"className":40710,"style":7767},[508,581],[47,40712,40713,40716],{"style":663},[47,40714],{"className":40715,"style":538},[537],[47,40717],{"className":40718,"style":671},[670],[47,40720,40721,40724],{"style":16247},[47,40722],{"className":40723,"style":538},[537],[47,40725,40727],{"className":40726},[508],[47,40728,472],{"className":40729},[508],[47,40731,616],{"className":40732},[615],[47,40734,40736],{"className":40735},[525],[47,40737,40739],{"className":40738,"style":16681},[529],[47,40740],{},[47,40742],{"className":40743},[702,513],[47,40745],{"className":40746,"style":1303},[888],[47,40748,40750],{"className":40749},[1338,14600],[47,40751,40753,40806],{"className":40752},[520,521],[47,40754,40756,40803],{"className":40755},[525],[47,40757,40759,40779,40789],{"className":40758,"style":16702},[529],[47,40760,40761,40764],{"style":16705},[47,40762],{"className":40763,"style":15794},[537],[47,40765,40767],{"className":40766},[542,543,544,545],[47,40768,40770,40773,40776],{"className":40769},[508,545],[47,40771,724],{"className":40772},[508,581,545],[47,40774,82],{"className":40775},[893,545],[47,40777,472],{"className":40778},[508,545],[47,40780,40781,40784],{"style":15803},[47,40782],{"className":40783,"style":15794},[537],[47,40785,40786],{},[47,40787,2213],{"className":40788},[1338,2297,15812],[47,40790,40791,40794],{"style":16736},[47,40792],{"className":40793,"style":15794},[537],[47,40795,40797],{"className":40796},[542,543,544,545],[47,40798,40800],{"className":40799},[508,545],[47,40801,7692],{"className":40802,"style":7767},[508,581,545],[47,40804,616],{"className":40805},[615],[47,40807,40809],{"className":40808},[525],[47,40810,40812],{"className":40811,"style":16755},[529],[47,40813],{},[47,40815],{"className":40816,"style":1303},[888],[47,40818,40820],{"className":40819},[1338,14600],[47,40821,40823,40913],{"className":40822},[520,521],[47,40824,40826,40910],{"className":40825},[525],[47,40827,40829,40849,40859],{"className":40828,"style":17511},[529],[47,40830,40831,40834],{"style":16705},[47,40832],{"className":40833,"style":15794},[537],[47,40835,40837],{"className":40836},[542,543,544,545],[47,40838,40840,40843,40846],{"className":40839},[508,545],[47,40841,1166],{"className":40842},[508,581,545],[47,40844,82],{"className":40845},[893,545],[47,40847,472],{"className":40848},[508,545],[47,40850,40851,40854],{"style":15803},[47,40852],{"className":40853,"style":15794},[537],[47,40855,40856],{},[47,40857,2213],{"className":40858},[1338,2297,15812],[47,40860,40861,40864],{"style":17544},[47,40862],{"className":40863,"style":15794},[537],[47,40865,40867],{"className":40866},[542,543,544,545],[47,40868,40870],{"className":40869},[508,545],[47,40871,40873,40876],{"className":40872},[508,545],[47,40874,13604],{"className":40875,"style":3555},[508,581,545],[47,40877,40879],{"className":40878},[585],[47,40880,40882,40902],{"className":40881},[520,521],[47,40883,40885,40899],{"className":40884},[525],[47,40886,40888],{"className":40887,"style":4672},[529],[47,40889,40890,40893],{"style":17574},[47,40891],{"className":40892,"style":602},[537],[47,40894,40896],{"className":40895},[542,606,607,545],[47,40897,724],{"className":40898},[508,581,545],[47,40900,616],{"className":40901},[615],[47,40903,40905],{"className":40904},[525],[47,40906,40908],{"className":40907,"style":4694},[529],[47,40909],{},[47,40911,616],{"className":40912},[615],[47,40914,40916],{"className":40915},[525],[47,40917,40919],{"className":40918,"style":16755},[529],[47,40920],{},[47,40922],{"className":40923,"style":1303},[888],[47,40925,40927,40930],{"className":40926},[508],[47,40928,1163],{"className":40929},[508,581],[47,40931,40933],{"className":40932},[585],[47,40934,40936,41008],{"className":40935},[520,521],[47,40937,40939,41005],{"className":40938},[525],[47,40940,40942,40985],{"className":40941,"style":25158},[529],[47,40943,40944,40947],{"style":17673},[47,40945],{"className":40946,"style":929},[537],[47,40948,40950],{"className":40949},[542,543,544,545],[47,40951,40953],{"className":40952},[508,545],[47,40954,40956,40959],{"className":40955},[508,545],[47,40957,14413],{"className":40958,"style":3580},[508,581,545],[47,40960,40962],{"className":40961},[585],[47,40963,40965],{"className":40964},[520],[47,40966,40968],{"className":40967},[525],[47,40969,40971],{"className":40970,"style":19506},[529],[47,40972,40973,40976],{"style":19509},[47,40974],{"className":40975,"style":602},[537],[47,40977,40979],{"className":40978},[542,606,607,545],[47,40980,40982],{"className":40981},[508,545],[47,40983,19346],{"className":40984},[508,545],[47,40986,40987,40990],{"style":16788},[47,40988],{"className":40989,"style":929},[537],[47,40991,40993],{"className":40992},[542,543,544,545],[47,40994,40996,40999,41002],{"className":40995},[508,545],[47,40997,27872],{"className":40998},[508,581,545],[47,41000,1163],{"className":41001},[508,581,545],[47,41003,5977],{"className":41004,"style":6420},[508,581,545],[47,41006,616],{"className":41007},[615],[47,41009,41011],{"className":41010},[525],[47,41012,41014],{"className":41013,"style":17703},[529],[47,41015],{},[47,41017,1187],{"className":41018},[512],[47,41020,41022,41025],{"className":41021},[508],[47,41023,13164],{"className":41024},[508,581],[47,41026,41028],{"className":41027},[585],[47,41029,41031,41062],{"className":41030},[520,521],[47,41032,41034,41059],{"className":41033},[525],[47,41035,41037,41048],{"className":41036,"style":17670},[529],[47,41038,41039,41042],{"style":17673},[47,41040],{"className":41041,"style":929},[537],[47,41043,41045],{"className":41044},[542,543,544,545],[47,41046,724],{"className":41047},[508,581,545],[47,41049,41050,41053],{"style":16788},[47,41051],{"className":41052,"style":929},[537],[47,41054,41056],{"className":41055},[542,543,544,545],[47,41057,1166],{"className":41058},[508,581,545],[47,41060,616],{"className":41061},[615],[47,41063,41065],{"className":41064},[525],[47,41066,41068],{"className":41067,"style":17703},[529],[47,41069],{},[47,41071,1192],{"className":41072},[1299],[47,41074],{"className":41075,"style":1303},[888],[47,41077,41079,41082],{"className":41078},[508],[47,41080,285],{"className":41081},[508,581],[47,41083,41085],{"className":41084},[585],[47,41086,41088,41119],{"className":41087},[520,521],[47,41089,41091,41116],{"className":41090},[525],[47,41092,41094,41105],{"className":41093,"style":17670},[529],[47,41095,41096,41099],{"style":17673},[47,41097],{"className":41098,"style":929},[537],[47,41100,41102],{"className":41101},[542,543,544,545],[47,41103,724],{"className":41104},[508,581,545],[47,41106,41107,41110],{"style":16788},[47,41108],{"className":41109,"style":929},[537],[47,41111,41113],{"className":41112},[542,543,544,545],[47,41114,1166],{"className":41115},[508,581,545],[47,41117,616],{"className":41118},[615],[47,41120,41122],{"className":41121},[525],[47,41123,41125],{"className":41124,"style":17703},[529],[47,41126],{},[47,41128,1201],{"className":41129},[702],[47,41131,41133,41136,41520],{"className":41132},[508],[47,41134],{"className":41135},[512,513],[47,41137,41139],{"className":41138},[467],[47,41140,41142,41512],{"className":41141},[520,521],[47,41143,41145,41509],{"className":41144},[525],[47,41146,41148,41339,41347],{"className":41147,"style":36665},[529],[47,41149,41150,41153],{"style":16181},[47,41151],{"className":41152,"style":538},[537],[47,41154,41156,41228,41231,41282,41285,41336],{"className":41155},[508],[47,41157,41159,41162],{"className":41158},[508],[47,41160,5974],{"className":41161,"style":3555},[508,581],[47,41163,41165],{"className":41164},[585],[47,41166,41168,41220],{"className":41167},[520,521],[47,41169,41171,41217],{"className":41170},[525],[47,41172,41174],{"className":41173,"style":1635},[529],[47,41175,41176,41179],{"style":7830},[47,41177],{"className":41178,"style":929},[537],[47,41180,41182],{"className":41181},[542,543,544,545],[47,41183,41185],{"className":41184},[508,545],[47,41186,41188,41191],{"className":41187},[508,545],[47,41189,14413],{"className":41190,"style":3580},[508,581,545],[47,41192,41194],{"className":41193},[585],[47,41195,41197],{"className":41196},[520],[47,41198,41200],{"className":41199},[525],[47,41201,41203],{"className":41202,"style":19506},[529],[47,41204,41205,41208],{"style":19509},[47,41206],{"className":41207,"style":602},[537],[47,41209,41211],{"className":41210},[542,606,607,545],[47,41212,41214],{"className":41213},[508,545],[47,41215,19346],{"className":41216},[508,545],[47,41218,616],{"className":41219},[615],[47,41221,41223],{"className":41222},[525],[47,41224,41226],{"className":41225,"style":1657},[529],[47,41227],{},[47,41229,1187],{"className":41230},[512],[47,41232,41234,41237],{"className":41233},[508],[47,41235,285],{"className":41236},[508,581],[47,41238,41240],{"className":41239},[585],[47,41241,41243,41274],{"className":41242},[520,521],[47,41244,41246,41271],{"className":41245},[525],[47,41247,41249,41260],{"className":41248,"style":36767},[529],[47,41250,41251,41254],{"style":17673},[47,41252],{"className":41253,"style":929},[537],[47,41255,41257],{"className":41256},[542,543,544,545],[47,41258,724],{"className":41259},[508,581,545],[47,41261,41262,41265],{"style":16891},[47,41263],{"className":41264,"style":929},[537],[47,41266,41268],{"className":41267},[542,543,544,545],[47,41269,1166],{"className":41270},[508,581,545],[47,41272,616],{"className":41273},[615],[47,41275,41277],{"className":41276},[525],[47,41278,41280],{"className":41279,"style":17703},[529],[47,41281],{},[47,41283,13303],{"className":41284},[508],[47,41286,41288,41291],{"className":41287},[508],[47,41289,13164],{"className":41290},[508,581],[47,41292,41294],{"className":41293},[585],[47,41295,41297,41328],{"className":41296},[520,521],[47,41298,41300,41325],{"className":41299},[525],[47,41301,41303,41314],{"className":41302,"style":36767},[529],[47,41304,41305,41308],{"style":17673},[47,41306],{"className":41307,"style":929},[537],[47,41309,41311],{"className":41310},[542,543,544,545],[47,41312,724],{"className":41313},[508,581,545],[47,41315,41316,41319],{"style":16891},[47,41317],{"className":41318,"style":929},[537],[47,41320,41322],{"className":41321},[542,543,544,545],[47,41323,1166],{"className":41324},[508,581,545],[47,41326,616],{"className":41327},[615],[47,41329,41331],{"className":41330},[525],[47,41332,41334],{"className":41333,"style":17703},[529],[47,41335],{},[47,41337,1201],{"className":41338},[702],[47,41340,41341,41344],{"style":663},[47,41342],{"className":41343,"style":538},[537],[47,41345],{"className":41346,"style":671},[670],[47,41348,41349,41352],{"style":16247},[47,41350],{"className":41351,"style":538},[537],[47,41353,41355,41398,41401,41452,41455,41506],{"className":41354},[508],[47,41356,41358,41361],{"className":41357},[508],[47,41359,5974],{"className":41360,"style":3555},[508,581],[47,41362,41364],{"className":41363},[585],[47,41365,41367,41390],{"className":41366},[520,521],[47,41368,41370,41387],{"className":41369},[525],[47,41371,41373],{"className":41372,"style":1635},[529],[47,41374,41375,41378],{"style":7830},[47,41376],{"className":41377,"style":929},[537],[47,41379,41381],{"className":41380},[542,543,544,545],[47,41382,41384],{"className":41383},[508,545],[47,41385,14413],{"className":41386,"style":3580},[508,581,545],[47,41388,616],{"className":41389},[615],[47,41391,41393],{"className":41392},[525],[47,41394,41396],{"className":41395,"style":1657},[529],[47,41397],{},[47,41399,1187],{"className":41400},[512],[47,41402,41404,41407],{"className":41403},[508],[47,41405,285],{"className":41406},[508,581],[47,41408,41410],{"className":41409},[585],[47,41411,41413,41444],{"className":41412},[520,521],[47,41414,41416,41441],{"className":41415},[525],[47,41417,41419,41430],{"className":41418,"style":19211},[529],[47,41420,41421,41424],{"style":17673},[47,41422],{"className":41423,"style":929},[537],[47,41425,41427],{"className":41426},[542,543,544,545],[47,41428,724],{"className":41429},[508,581,545],[47,41431,41432,41435],{"style":925},[47,41433],{"className":41434,"style":929},[537],[47,41436,41438],{"className":41437},[542,543,544,545],[47,41439,1166],{"className":41440},[508,581,545],[47,41442,616],{"className":41443},[615],[47,41445,41447],{"className":41446},[525],[47,41448,41450],{"className":41449,"style":17703},[529],[47,41451],{},[47,41453,13303],{"className":41454},[508],[47,41456,41458,41461],{"className":41457},[508],[47,41459,13164],{"className":41460},[508,581],[47,41462,41464],{"className":41463},[585],[47,41465,41467,41498],{"className":41466},[520,521],[47,41468,41470,41495],{"className":41469},[525],[47,41471,41473,41484],{"className":41472,"style":19211},[529],[47,41474,41475,41478],{"style":17673},[47,41476],{"className":41477,"style":929},[537],[47,41479,41481],{"className":41480},[542,543,544,545],[47,41482,724],{"className":41483},[508,581,545],[47,41485,41486,41489],{"style":925},[47,41487],{"className":41488,"style":929},[537],[47,41490,41492],{"className":41491},[542,543,544,545],[47,41493,1166],{"className":41494},[508,581,545],[47,41496,616],{"className":41497},[615],[47,41499,41501],{"className":41500},[525],[47,41502,41504],{"className":41503,"style":17703},[529],[47,41505],{},[47,41507,1201],{"className":41508},[702],[47,41510,616],{"className":41511},[615],[47,41513,41515],{"className":41514},[525],[47,41516,41518],{"className":41517,"style":16312},[529],[47,41519],{},[47,41521],{"className":41522},[702,513],[47,41524],{"className":41525,"style":2259},[888],[47,41527,6147],{"className":41528},[942],[47,41530],{"className":41531,"style":2259},[888],[47,41533,41535,41538,41541,41544,41551,41554,41597,41600,41603,41675],{"className":41534},[499],[47,41536],{"className":41537,"style":1264},[503],[47,41539,40368],{"className":41540,"style":40385},[508,581],[47,41542],{"className":41543,"style":1303},[888],[47,41545,41547],{"className":41546},[508],[47,41548,41550],{"className":41549},[508,1342],"KL",[47,41552,1187],{"className":41553},[512],[47,41555,41557,41560],{"className":41556},[508],[47,41558,5974],{"className":41559,"style":3555},[508,581],[47,41561,41563],{"className":41562},[585],[47,41564,41566,41589],{"className":41565},[520,521],[47,41567,41569,41586],{"className":41568},[525],[47,41570,41572],{"className":41571,"style":1635},[529],[47,41573,41574,41577],{"style":7830},[47,41575],{"className":41576,"style":929},[537],[47,41578,41580],{"className":41579},[542,543,544,545],[47,41581,41583],{"className":41582},[508,545],[47,41584,14413],{"className":41585,"style":3580},[508,581,545],[47,41587,616],{"className":41588},[615],[47,41590,41592],{"className":41591},[525],[47,41593,41595],{"className":41594,"style":1657},[529],[47,41596],{},[47,41598,1192],{"className":41599},[1299],[47,41601],{"className":41602,"style":1303},[888],[47,41604,41606,41609],{"className":41605},[508],[47,41607,5974],{"className":41608,"style":3555},[508,581],[47,41610,41612],{"className":41611},[585],[47,41613,41615,41667],{"className":41614},[520,521],[47,41616,41618,41664],{"className":41617},[525],[47,41619,41621],{"className":41620,"style":1635},[529],[47,41622,41623,41626],{"style":7830},[47,41624],{"className":41625,"style":929},[537],[47,41627,41629],{"className":41628},[542,543,544,545],[47,41630,41632],{"className":41631},[508,545],[47,41633,41635,41638],{"className":41634},[508,545],[47,41636,14413],{"className":41637,"style":3580},[508,581,545],[47,41639,41641],{"className":41640},[585],[47,41642,41644],{"className":41643},[520],[47,41645,41647],{"className":41646},[525],[47,41648,41650],{"className":41649,"style":19506},[529],[47,41651,41652,41655],{"style":19509},[47,41653],{"className":41654,"style":602},[537],[47,41656,41658],{"className":41657},[542,606,607,545],[47,41659,41661],{"className":41660},[508,545],[47,41662,19346],{"className":41663},[508,545],[47,41665,616],{"className":41666},[615],[47,41668,41670],{"className":41669},[525],[47,41671,41673],{"className":41672,"style":1657},[529],[47,41674],{},[47,41676,1201],{"className":41677},[702],[40347,41679,41681],{"id":41680},"ppo-clip截断重要性采样比","PPO-Clip：截断重要性采样比",[14,41683,41684],{},"PPO 还有一种实现，是通过截断函数来替代 KL 散度，防止训练的策略和参考的策略偏差过大，可以看出它是由红色和蓝色两个部分构成，取两个部分的最小值。",[14,41686,41687,41688,42188,42189,7131,42242,42293],{},"其中红色部分是原始公式，蓝色部分是一个截断函数，截断函数内由三个部分组成。如果 ",[47,41689,41691,41769],{"className":41690},[451],[47,41692,41694],{"className":41693},[455],[457,41695,41696],{"xmlns":459},[461,41697,41698,41766],{},[464,41699,41700],{},[467,41701,41702,41732],{},[464,41703,41704,41710,41712,41720,41722,41730],{},[477,41705,41706,41708],{},[480,41707,5974],{},[480,41709,14413],{},[850,41711,1187],{"stretchy":1186},[2209,41713,41714,41716,41718],{},[480,41715,285],{},[480,41717,724],{},[480,41719,1166],{},[480,41721,13303],{"mathvariant":1206},[2209,41723,41724,41726,41728],{},[480,41725,13164],{},[480,41727,724],{},[480,41729,1166],{},[850,41731,1201],{"stretchy":1186},[464,41733,41734,41744,41746,41754,41756,41764],{},[477,41735,41736,41738],{},[480,41737,5974],{},[854,41739,41740,41742],{},[480,41741,14413],{},[850,41743,19346],{"mathvariant":1206,"lspace":14728,"rspace":14728},[850,41745,1187],{"stretchy":1186},[2209,41747,41748,41750,41752],{},[480,41749,285],{},[480,41751,724],{},[480,41753,1166],{},[480,41755,13303],{"mathvariant":1206},[2209,41757,41758,41760,41762],{},[480,41759,13164],{},[480,41761,724],{},[480,41763,1166],{},[850,41765,1201],{"stretchy":1186},[487,41767,41768],{"encoding":489},"\\frac{P_{\\theta}(a_n^t|s_n^t)}{P_{\\theta'}(a_n^t|s_n^t)}",[47,41770,41772],{"className":41771,"ariaHidden":495},[494],[47,41773,41775,41779],{"className":41774},[499],[47,41776],{"className":41777,"style":41778},[503],"height:1.6646em;vertical-align:-0.5452em;",[47,41780,41782,41785,42185],{"className":41781},[508],[47,41783],{"className":41784},[512,513],[47,41786,41788],{"className":41787},[467],[47,41789,41791,42176],{"className":41790},[520,521],[47,41792,41794,42173],{"className":41793},[525],[47,41795,41798,41998,42006],{"className":41796,"style":41797},[529],"height:1.1194em;",[47,41799,41800,41803],{"style":18868},[47,41801],{"className":41802,"style":538},[537],[47,41804,41806],{"className":41805},[542,543,544,545],[47,41807,41809,41884,41887,41941,41944,41995],{"className":41808},[508,545],[47,41810,41812,41815],{"className":41811},[508,545],[47,41813,5974],{"className":41814,"style":3555},[508,581,545],[47,41816,41818],{"className":41817},[585],[47,41819,41821,41875],{"className":41820},[520,521],[47,41822,41824,41872],{"className":41823},[525],[47,41825,41827],{"className":41826,"style":595},[529],[47,41828,41830,41834],{"style":41829},"top:-2.3448em;margin-left:-0.1389em;margin-right:0.0714em;",[47,41831],{"className":41832,"style":41833},[537],"height:2.6068em;",[47,41835,41837],{"className":41836},[542,606,607,545],[47,41838,41840],{"className":41839},[508,545],[47,41841,41843,41846],{"className":41842},[508,545],[47,41844,14413],{"className":41845,"style":3580},[508,581,545],[47,41847,41849],{"className":41848},[585],[47,41850,41852],{"className":41851},[520],[47,41853,41855],{"className":41854},[525],[47,41856,41859],{"className":41857,"style":41858},[529],"height:0.8496em;",[47,41860,41862,41866],{"style":41861},"top:-2.8496em;margin-right:0.1em;",[47,41863],{"className":41864,"style":41865},[537],"height:2.5556em;",[47,41867,41869],{"className":41868},[508,545],[47,41870,19346],{"className":41871},[508,545],[47,41873,616],{"className":41874},[615],[47,41876,41878],{"className":41877},[525],[47,41879,41882],{"className":41880,"style":41881},[529],"height:0.262em;",[47,41883],{},[47,41885,1187],{"className":41886},[512,545],[47,41888,41890,41893],{"className":41889},[508,545],[47,41891,285],{"className":41892},[508,581,545],[47,41894,41896],{"className":41895},[585],[47,41897,41899,41932],{"className":41898},[520,521],[47,41900,41902,41929],{"className":41901},[525],[47,41903,41906,41918],{"className":41904,"style":41905},[529],"height:0.7253em;",[47,41907,41909,41912],{"style":41908},"top:-2.214em;margin-left:0em;margin-right:0.0714em;",[47,41910],{"className":41911,"style":602},[537],[47,41913,41915],{"className":41914},[542,606,607,545],[47,41916,724],{"className":41917},[508,581,545],[47,41919,41920,41923],{"style":19509},[47,41921],{"className":41922,"style":602},[537],[47,41924,41926],{"className":41925},[542,606,607,545],[47,41927,1166],{"className":41928},[508,581,545],[47,41930,616],{"className":41931},[615],[47,41933,41935],{"className":41934},[525],[47,41936,41939],{"className":41937,"style":41938},[529],"height:0.286em;",[47,41940],{},[47,41942,13303],{"className":41943},[508,545],[47,41945,41947,41950],{"className":41946},[508,545],[47,41948,13164],{"className":41949},[508,581,545],[47,41951,41953],{"className":41952},[585],[47,41954,41956,41987],{"className":41955},[520,521],[47,41957,41959,41984],{"className":41958},[525],[47,41960,41962,41973],{"className":41961,"style":41905},[529],[47,41963,41964,41967],{"style":41908},[47,41965],{"className":41966,"style":602},[537],[47,41968,41970],{"className":41969},[542,606,607,545],[47,41971,724],{"className":41972},[508,581,545],[47,41974,41975,41978],{"style":19509},[47,41976],{"className":41977,"style":602},[537],[47,41979,41981],{"className":41980},[542,606,607,545],[47,41982,1166],{"className":41983},[508,581,545],[47,41985,616],{"className":41986},[615],[47,41988,41990],{"className":41989},[525],[47,41991,41993],{"className":41992,"style":41938},[529],[47,41994],{},[47,41996,1201],{"className":41997},[702,545],[47,41999,42000,42003],{"style":663},[47,42001],{"className":42002,"style":538},[537],[47,42004],{"className":42005,"style":671},[670],[47,42007,42009,42012],{"style":42008},"top:-3.5102em;",[47,42010],{"className":42011,"style":538},[537],[47,42013,42015],{"className":42014},[542,543,544,545],[47,42016,42018,42061,42064,42116,42119,42170],{"className":42017},[508,545],[47,42019,42021,42024],{"className":42020},[508,545],[47,42022,5974],{"className":42023,"style":3555},[508,581,545],[47,42025,42027],{"className":42026},[585],[47,42028,42030,42053],{"className":42029},[520,521],[47,42031,42033,42050],{"className":42032},[525],[47,42034,42036],{"className":42035,"style":595},[529],[47,42037,42038,42041],{"style":14537},[47,42039],{"className":42040,"style":602},[537],[47,42042,42044],{"className":42043},[542,606,607,545],[47,42045,42047],{"className":42046},[508,545],[47,42048,14413],{"className":42049,"style":3580},[508,581,545],[47,42051,616],{"className":42052},[615],[47,42054,42056],{"className":42055},[525],[47,42057,42059],{"className":42058,"style":623},[529],[47,42060],{},[47,42062,1187],{"className":42063},[512,545],[47,42065,42067,42070],{"className":42066},[508,545],[47,42068,285],{"className":42069},[508,581,545],[47,42071,42073],{"className":42072},[585],[47,42074,42076,42108],{"className":42075},[520,521],[47,42077,42079,42105],{"className":42078},[525],[47,42080,42083,42094],{"className":42081,"style":42082},[529],"height:0.8703em;",[47,42084,42085,42088],{"style":41908},[47,42086],{"className":42087,"style":602},[537],[47,42089,42091],{"className":42090},[542,606,607,545],[47,42092,724],{"className":42093},[508,581,545],[47,42095,42096,42099],{"style":1485},[47,42097],{"className":42098,"style":602},[537],[47,42100,42102],{"className":42101},[542,606,607,545],[47,42103,1166],{"className":42104},[508,581,545],[47,42106,616],{"className":42107},[615],[47,42109,42111],{"className":42110},[525],[47,42112,42114],{"className":42113,"style":41938},[529],[47,42115],{},[47,42117,13303],{"className":42118},[508,545],[47,42120,42122,42125],{"className":42121},[508,545],[47,42123,13164],{"className":42124},[508,581,545],[47,42126,42128],{"className":42127},[585],[47,42129,42131,42162],{"className":42130},[520,521],[47,42132,42134,42159],{"className":42133},[525],[47,42135,42137,42148],{"className":42136,"style":42082},[529],[47,42138,42139,42142],{"style":41908},[47,42140],{"className":42141,"style":602},[537],[47,42143,42145],{"className":42144},[542,606,607,545],[47,42146,724],{"className":42147},[508,581,545],[47,42149,42150,42153],{"style":1485},[47,42151],{"className":42152,"style":602},[537],[47,42154,42156],{"className":42155},[542,606,607,545],[47,42157,1166],{"className":42158},[508,581,545],[47,42160,616],{"className":42161},[615],[47,42163,42165],{"className":42164},[525],[47,42166,42168],{"className":42167,"style":41938},[529],[47,42169],{},[47,42171,1201],{"className":42172},[702,545],[47,42174,616],{"className":42175},[615],[47,42177,42179],{"className":42178},[525],[47,42180,42183],{"className":42181,"style":42182},[529],"height:0.5452em;",[47,42184],{},[47,42186],{"className":42187},[702,513]," 的值介于 ",[47,42190,42192,42211],{"className":42191},[451],[47,42193,42195],{"className":42194},[455],[457,42196,42197],{"xmlns":459},[461,42198,42199,42208],{},[464,42200,42201,42203,42205],{},[470,42202,472],{},[850,42204,18727],{},[480,42206,42207],{},"ϵ",[487,42209,42210],{"encoding":489},"1-\\epsilon",[47,42212,42214,42233],{"className":42213,"ariaHidden":495},[494],[47,42215,42217,42221,42224,42227,42230],{"className":42216},[499],[47,42218],{"className":42219,"style":42220},[503],"height:0.7278em;vertical-align:-0.0833em;",[47,42222,472],{"className":42223},[508],[47,42225],{"className":42226,"style":2259},[888],[47,42228,18727],{"className":42229},[942],[47,42231],{"className":42232,"style":2259},[888],[47,42234,42236,42239],{"className":42235},[499],[47,42237],{"className":42238,"style":736},[503],[47,42240,42207],{"className":42241},[508,581],[47,42243,42245,42263],{"className":42244},[451],[47,42246,42248],{"className":42247},[455],[457,42249,42250],{"xmlns":459},[461,42251,42252,42260],{},[464,42253,42254,42256,42258],{},[470,42255,472],{},[850,42257,6147],{},[480,42259,42207],{},[487,42261,42262],{"encoding":489},"1+\\epsilon",[47,42264,42266,42284],{"className":42265,"ariaHidden":495},[494],[47,42267,42269,42272,42275,42278,42281],{"className":42268},[499],[47,42270],{"className":42271,"style":42220},[503],[47,42273,472],{"className":42274},[508],[47,42276],{"className":42277,"style":2259},[888],[47,42279,6147],{"className":42280},[942],[47,42282],{"className":42283,"style":2259},[888],[47,42285,42287,42290],{"className":42286},[499],[47,42288],{"className":42289,"style":736},[503],[47,42291,42207],{"className":42292},[508,581]," 之间，就返回原值，否则返回其中更接近的一个。这就限制的参考模型和训练模型差异的值不能太大。",[47,42295,42297],{"className":42296},[14712],[47,42298,42300,42636],{"className":42299},[451],[47,42301,42303],{"className":42302},[455],[457,42304,42305],{"xmlns":459,"display":9848},[461,42306,42307,42633],{},[464,42308,42309,42311,42313,42315,42322,42324,42326,42332,42346,42364,42367,42369],{},[480,42310,8186],{},[480,42312,1181],{},[480,42314,13164],{},[477,42316,42317,42319],{},[480,42318,13164],{},[3647,42320,42321],{},"ppo2",[850,42323,82],{},[850,42325,18727],{},[467,42327,42328,42330],{},[470,42329,472],{},[480,42331,7692],{},[15045,42333,42334,42336,42344],{},[850,42335,2213],{},[464,42337,42338,42340,42342],{},[480,42339,724],{},[850,42341,82],{},[470,42343,472],{},[480,42345,7692],{},[15045,42347,42348,42350,42358],{},[850,42349,2213],{},[464,42351,42352,42354,42356],{},[480,42353,1166],{},[850,42355,82],{},[470,42357,472],{},[477,42359,42360,42362],{},[480,42361,13604],{},[480,42363,724],{},[480,42365,42366],{},"min",[850,42368,1210],{},[464,42370,42371,42373,42482,42484,42486,42631],{},[850,42372,1187],{"fence":495},[14736,42374,42376,42394,42396,42404,42406,42414,42416],{"mathcolor":42375},"red",[2209,42377,42378,42380,42386],{},[480,42379,1163],{},[854,42381,42382,42384],{},[480,42383,14413],{},[850,42385,19346],{"mathvariant":1206,"lspace":14728,"rspace":14728},[464,42387,42388,42390,42392],{},[480,42389,27872],{},[480,42391,1163],{},[480,42393,5977],{},[850,42395,1187],{"stretchy":1186},[2209,42397,42398,42400,42402],{},[480,42399,13164],{},[480,42401,724],{},[480,42403,1166],{},[850,42405,1192],{"separator":495},[2209,42407,42408,42410,42412],{},[480,42409,285],{},[480,42411,724],{},[480,42413,1166],{},[850,42415,1201],{"stretchy":1186},[467,42417,42418,42448],{},[464,42419,42420,42426,42428,42436,42438,42446],{},[477,42421,42422,42424],{},[480,42423,5974],{},[480,42425,14413],{},[850,42427,1187],{"stretchy":1186},[2209,42429,42430,42432,42434],{},[480,42431,285],{},[480,42433,724],{},[480,42435,1166],{},[480,42437,13303],{"mathvariant":1206},[2209,42439,42440,42442,42444],{},[480,42441,13164],{},[480,42443,724],{},[480,42445,1166],{},[850,42447,1201],{"stretchy":1186},[464,42449,42450,42460,42462,42470,42472,42480],{},[477,42451,42452,42454],{},[480,42453,5974],{},[854,42455,42456,42458],{},[480,42457,14413],{},[850,42459,19346],{"mathvariant":1206,"lspace":14728,"rspace":14728},[850,42461,1187],{"stretchy":1186},[2209,42463,42464,42466,42468],{},[480,42465,285],{},[480,42467,724],{},[480,42469,1166],{},[480,42471,13303],{"mathvariant":1206},[2209,42473,42474,42476,42478],{},[480,42475,13164],{},[480,42477,724],{},[480,42479,1166],{},[850,42481,1201],{"stretchy":1186},[850,42483,1192],{"separator":495},[3647,42485,27957],{},[14736,42487,42489,42499,42591,42609,42611,42619,42621,42629],{"mathcolor":42488},"skyblue",[464,42490,42491,42493,42495,42497],{},[480,42492,3314],{"mathvariant":1206},[480,42494,3273],{"mathvariant":1206},[480,42496,1178],{"mathvariant":1206},[480,42498,14],{"mathvariant":1206},[464,42500,42501,42503,42569,42571,42573,42575,42577,42579,42581,42583,42585,42587,42589],{},[850,42502,1187],{"fence":495},[467,42504,42505,42535],{},[464,42506,42507,42513,42515,42523,42525,42533],{},[477,42508,42509,42511],{},[480,42510,5974],{},[480,42512,14413],{},[850,42514,1187],{"stretchy":1186},[2209,42516,42517,42519,42521],{},[480,42518,285],{},[480,42520,724],{},[480,42522,1166],{},[480,42524,13303],{"mathvariant":1206},[2209,42526,42527,42529,42531],{},[480,42528,13164],{},[480,42530,724],{},[480,42532,1166],{},[850,42534,1201],{"stretchy":1186},[464,42536,42537,42547,42549,42557,42559,42567],{},[477,42538,42539,42541],{},[480,42540,5974],{},[854,42542,42543,42545],{},[480,42544,14413],{},[850,42546,19346],{"mathvariant":1206,"lspace":14728,"rspace":14728},[850,42548,1187],{"stretchy":1186},[2209,42550,42551,42553,42555],{},[480,42552,285],{},[480,42554,724],{},[480,42556,1166],{},[480,42558,13303],{"mathvariant":1206},[2209,42560,42561,42563,42565],{},[480,42562,13164],{},[480,42564,724],{},[480,42566,1166],{},[850,42568,1201],{"stretchy":1186},[850,42570,1192],{"separator":495},[3647,42572,27957],{},[470,42574,472],{},[850,42576,18727],{},[480,42578,42207],{},[850,42580,1192],{"separator":495},[3647,42582,27957],{},[470,42584,472],{},[850,42586,6147],{},[480,42588,42207],{},[850,42590,1201],{"fence":495},[2209,42592,42593,42595,42601],{},[480,42594,1163],{},[854,42596,42597,42599],{},[480,42598,14413],{},[850,42600,19346],{"mathvariant":1206,"lspace":14728,"rspace":14728},[464,42602,42603,42605,42607],{},[480,42604,27872],{},[480,42606,1163],{},[480,42608,5977],{},[850,42610,1187],{"stretchy":1186},[2209,42612,42613,42615,42617],{},[480,42614,13164],{},[480,42616,724],{},[480,42618,1166],{},[850,42620,1192],{"separator":495},[2209,42622,42623,42625,42627],{},[480,42624,285],{},[480,42626,724],{},[480,42628,1166],{},[850,42630,1201],{"stretchy":1186},[850,42632,1201],{"fence":495},[487,42634,42635],{"encoding":489},"Loss_{\\text{ppo2}} = -\\frac{1}{N} \\sum_{n=1}^{N} \\sum_{t=1}^{T_n} \\min\\left(\\textcolor{red}{A_{\\theta'}^{GAE}(s_n^t, a_n^t) \\frac{P_{\\theta}(a_n^t|s_n^t)}{P_{\\theta'}(a_n^t|s_n^t)}},\\,\\textcolor{skyblue}{\\mathrm{clip}\\left(\\frac{P_{\\theta}(a_n^t|s_n^t)}{P_{\\theta'}(a_n^t|s_n^t)},\\, 1-\\epsilon,\\, 1+\\epsilon\\right) A_{\\theta'}^{GAE}(s_n^t, a_n^t)}\\right)",[47,42637,42639,42706],{"className":42638,"ariaHidden":495},[494],[47,42640,42642,42645,42648,42651,42697,42700,42703],{"className":42641},[499],[47,42643],{"className":42644,"style":40609},[503],[47,42646,8186],{"className":42647},[508,581],[47,42649,40616],{"className":42650},[508,581],[47,42652,42654,42657],{"className":42653},[508],[47,42655,13164],{"className":42656},[508,581],[47,42658,42660],{"className":42659},[585],[47,42661,42663,42689],{"className":42662},[520,521],[47,42664,42666,42686],{"className":42665},[525],[47,42667,42669],{"className":42668,"style":3461},[529],[47,42670,42671,42674],{"style":1638},[47,42672],{"className":42673,"style":929},[537],[47,42675,42677],{"className":42676},[542,543,544,545],[47,42678,42680],{"className":42679},[508,545],[47,42681,42683],{"className":42682},[508,12219,545],[47,42684,42321],{"className":42685},[508,545],[47,42687,616],{"className":42688},[615],[47,42690,42692],{"className":42691},[525],[47,42693,42695],{"className":42694,"style":40661},[529],[47,42696],{},[47,42698],{"className":42699,"style":889},[888],[47,42701,82],{"className":42702},[893],[47,42704],{"className":42705,"style":889},[888],[47,42707,42709,42712,42715,42777,42780,42847,42850,42954,42957,42960,42963],{"className":42708},[499],[47,42710],{"className":42711,"style":31535},[503],[47,42713,18727],{"className":42714},[508],[47,42716,42718,42721,42774],{"className":42717},[508],[47,42719],{"className":42720},[512,513],[47,42722,42724],{"className":42723},[467],[47,42725,42727,42766],{"className":42726},[520,521],[47,42728,42730,42763],{"className":42729},[525],[47,42731,42733,42744,42752],{"className":42732,"style":16641},[529],[47,42734,42735,42738],{"style":16181},[47,42736],{"className":42737,"style":538},[537],[47,42739,42741],{"className":42740},[508],[47,42742,7692],{"className":42743,"style":7767},[508,581],[47,42745,42746,42749],{"style":663},[47,42747],{"className":42748,"style":538},[537],[47,42750],{"className":42751,"style":671},[670],[47,42753,42754,42757],{"style":16247},[47,42755],{"className":42756,"style":538},[537],[47,42758,42760],{"className":42759},[508],[47,42761,472],{"className":42762},[508],[47,42764,616],{"className":42765},[615],[47,42767,42769],{"className":42768},[525],[47,42770,42772],{"className":42771,"style":16681},[529],[47,42773],{},[47,42775],{"className":42776},[702,513],[47,42778],{"className":42779,"style":1303},[888],[47,42781,42783],{"className":42782},[1338,14600],[47,42784,42786,42839],{"className":42785},[520,521],[47,42787,42789,42836],{"className":42788},[525],[47,42790,42792,42812,42822],{"className":42791,"style":16702},[529],[47,42793,42794,42797],{"style":16705},[47,42795],{"className":42796,"style":15794},[537],[47,42798,42800],{"className":42799},[542,543,544,545],[47,42801,42803,42806,42809],{"className":42802},[508,545],[47,42804,724],{"className":42805},[508,581,545],[47,42807,82],{"className":42808},[893,545],[47,42810,472],{"className":42811},[508,545],[47,42813,42814,42817],{"style":15803},[47,42815],{"className":42816,"style":15794},[537],[47,42818,42819],{},[47,42820,2213],{"className":42821},[1338,2297,15812],[47,42823,42824,42827],{"style":16736},[47,42825],{"className":42826,"style":15794},[537],[47,42828,42830],{"className":42829},[542,543,544,545],[47,42831,42833],{"className":42832},[508,545],[47,42834,7692],{"className":42835,"style":7767},[508,581,545],[47,42837,616],{"className":42838},[615],[47,42840,42842],{"className":42841},[525],[47,42843,42845],{"className":42844,"style":16755},[529],[47,42846],{},[47,42848],{"className":42849,"style":1303},[888],[47,42851,42853],{"className":42852},[1338,14600],[47,42854,42856,42946],{"className":42855},[520,521],[47,42857,42859,42943],{"className":42858},[525],[47,42860,42862,42882,42892],{"className":42861,"style":17511},[529],[47,42863,42864,42867],{"style":16705},[47,42865],{"className":42866,"style":15794},[537],[47,42868,42870],{"className":42869},[542,543,544,545],[47,42871,42873,42876,42879],{"className":42872},[508,545],[47,42874,1166],{"className":42875},[508,581,545],[47,42877,82],{"className":42878},[893,545],[47,42880,472],{"className":42881},[508,545],[47,42883,42884,42887],{"style":15803},[47,42885],{"className":42886,"style":15794},[537],[47,42888,42889],{},[47,42890,2213],{"className":42891},[1338,2297,15812],[47,42893,42894,42897],{"style":17544},[47,42895],{"className":42896,"style":15794},[537],[47,42898,42900],{"className":42899},[542,543,544,545],[47,42901,42903],{"className":42902},[508,545],[47,42904,42906,42909],{"className":42905},[508,545],[47,42907,13604],{"className":42908,"style":3555},[508,581,545],[47,42910,42912],{"className":42911},[585],[47,42913,42915,42935],{"className":42914},[520,521],[47,42916,42918,42932],{"className":42917},[525],[47,42919,42921],{"className":42920,"style":4672},[529],[47,42922,42923,42926],{"style":17574},[47,42924],{"className":42925,"style":602},[537],[47,42927,42929],{"className":42928},[542,606,607,545],[47,42930,724],{"className":42931},[508,581,545],[47,42933,616],{"className":42934},[615],[47,42936,42938],{"className":42937},[525],[47,42939,42941],{"className":42940,"style":4694},[529],[47,42942],{},[47,42944,616],{"className":42945},[615],[47,42947,42949],{"className":42948},[525],[47,42950,42952],{"className":42951,"style":16755},[529],[47,42953],{},[47,42955],{"className":42956,"style":1303},[888],[47,42958,42366],{"className":42959},[1338],[47,42961],{"className":42962,"style":1303},[888],[47,42964,42966,42972,43067,43070,43121,43124,43127,43178,43181,43576,43579,43582,43585,43593,43596,44062,44065,44158,44161,44212,44215,44218,44269,44272],{"className":42965},[1349],[47,42967,42969],{"className":42968,"style":1354},[512,1353],[47,42970,1187],{"className":42971},[1358,544],[47,42973,42976,42979],{"className":42974,"style":42975},[508],"color:red;",[47,42977,1163],{"className":42978,"style":42975},[508,581],[47,42980,42982],{"className":42981},[585],[47,42983,42985,43059],{"className":42984},[520,521],[47,42986,42988,43056],{"className":42987},[525],[47,42989,42991,43035],{"className":42990,"style":25158},[529],[47,42992,42993,42996],{"style":17673},[47,42994],{"className":42995,"style":929},[537],[47,42997,42999],{"className":42998,"style":42975},[542,543,544,545],[47,43000,43002],{"className":43001,"style":42975},[508,545],[47,43003,43005,43009],{"className":43004,"style":42975},[508,545],[47,43006,14413],{"className":43007,"style":43008},[508,581,545],"margin-right:0.02778em;color:red;",[47,43010,43012],{"className":43011},[585],[47,43013,43015],{"className":43014},[520],[47,43016,43018],{"className":43017},[525],[47,43019,43021],{"className":43020,"style":19506},[529],[47,43022,43023,43026],{"style":19509},[47,43024],{"className":43025,"style":602},[537],[47,43027,43029],{"className":43028,"style":42975},[542,606,607,545],[47,43030,43032],{"className":43031,"style":42975},[508,545],[47,43033,19346],{"className":43034,"style":42975},[508,545],[47,43036,43037,43040],{"style":16788},[47,43038],{"className":43039,"style":929},[537],[47,43041,43043],{"className":43042,"style":42975},[542,543,544,545],[47,43044,43046,43049,43052],{"className":43045,"style":42975},[508,545],[47,43047,27872],{"className":43048,"style":42975},[508,581,545],[47,43050,1163],{"className":43051,"style":42975},[508,581,545],[47,43053,5977],{"className":43054,"style":43055},[508,581,545],"margin-right:0.05764em;color:red;",[47,43057,616],{"className":43058},[615],[47,43060,43062],{"className":43061},[525],[47,43063,43065],{"className":43064,"style":17703},[529],[47,43066],{},[47,43068,1187],{"className":43069,"style":42975},[512],[47,43071,43073,43076],{"className":43072,"style":42975},[508],[47,43074,13164],{"className":43075,"style":42975},[508,581],[47,43077,43079],{"className":43078},[585],[47,43080,43082,43113],{"className":43081},[520,521],[47,43083,43085,43110],{"className":43084},[525],[47,43086,43088,43099],{"className":43087,"style":17670},[529],[47,43089,43090,43093],{"style":17673},[47,43091],{"className":43092,"style":929},[537],[47,43094,43096],{"className":43095,"style":42975},[542,543,544,545],[47,43097,724],{"className":43098,"style":42975},[508,581,545],[47,43100,43101,43104],{"style":16788},[47,43102],{"className":43103,"style":929},[537],[47,43105,43107],{"className":43106,"style":42975},[542,543,544,545],[47,43108,1166],{"className":43109,"style":42975},[508,581,545],[47,43111,616],{"className":43112},[615],[47,43114,43116],{"className":43115},[525],[47,43117,43119],{"className":43118,"style":17703},[529],[47,43120],{},[47,43122,1192],{"className":43123,"style":42975},[1299],[47,43125],{"className":43126,"style":1303},[888],[47,43128,43130,43133],{"className":43129,"style":42975},[508],[47,43131,285],{"className":43132,"style":42975},[508,581],[47,43134,43136],{"className":43135},[585],[47,43137,43139,43170],{"className":43138},[520,521],[47,43140,43142,43167],{"className":43141},[525],[47,43143,43145,43156],{"className":43144,"style":17670},[529],[47,43146,43147,43150],{"style":17673},[47,43148],{"className":43149,"style":929},[537],[47,43151,43153],{"className":43152,"style":42975},[542,543,544,545],[47,43154,724],{"className":43155,"style":42975},[508,581,545],[47,43157,43158,43161],{"style":16788},[47,43159],{"className":43160,"style":929},[537],[47,43162,43164],{"className":43163,"style":42975},[542,543,544,545],[47,43165,1166],{"className":43166,"style":42975},[508,581,545],[47,43168,616],{"className":43169},[615],[47,43171,43173],{"className":43172},[525],[47,43174,43176],{"className":43175,"style":17703},[529],[47,43177],{},[47,43179,1201],{"className":43180,"style":42975},[702],[47,43182,43184,43187,43573],{"className":43183,"style":42975},[508],[47,43185],{"className":43186},[512,513],[47,43188,43190],{"className":43189},[467],[47,43191,43193,43565],{"className":43192},[520,521],[47,43194,43196,43562],{"className":43195},[525],[47,43197,43199,43391,43400],{"className":43198,"style":36665},[529],[47,43200,43201,43204],{"style":16181},[47,43202],{"className":43203,"style":538},[537],[47,43205,43207,43280,43283,43334,43337,43388],{"className":43206,"style":42975},[508],[47,43208,43210,43214],{"className":43209,"style":42975},[508],[47,43211,5974],{"className":43212,"style":43213},[508,581],"margin-right:0.13889em;color:red;",[47,43215,43217],{"className":43216},[585],[47,43218,43220,43272],{"className":43219},[520,521],[47,43221,43223,43269],{"className":43222},[525],[47,43224,43226],{"className":43225,"style":1635},[529],[47,43227,43228,43231],{"style":7830},[47,43229],{"className":43230,"style":929},[537],[47,43232,43234],{"className":43233,"style":42975},[542,543,544,545],[47,43235,43237],{"className":43236,"style":42975},[508,545],[47,43238,43240,43243],{"className":43239,"style":42975},[508,545],[47,43241,14413],{"className":43242,"style":43008},[508,581,545],[47,43244,43246],{"className":43245},[585],[47,43247,43249],{"className":43248},[520],[47,43250,43252],{"className":43251},[525],[47,43253,43255],{"className":43254,"style":19506},[529],[47,43256,43257,43260],{"style":19509},[47,43258],{"className":43259,"style":602},[537],[47,43261,43263],{"className":43262,"style":42975},[542,606,607,545],[47,43264,43266],{"className":43265,"style":42975},[508,545],[47,43267,19346],{"className":43268,"style":42975},[508,545],[47,43270,616],{"className":43271},[615],[47,43273,43275],{"className":43274},[525],[47,43276,43278],{"className":43277,"style":1657},[529],[47,43279],{},[47,43281,1187],{"className":43282,"style":42975},[512],[47,43284,43286,43289],{"className":43285,"style":42975},[508],[47,43287,285],{"className":43288,"style":42975},[508,581],[47,43290,43292],{"className":43291},[585],[47,43293,43295,43326],{"className":43294},[520,521],[47,43296,43298,43323],{"className":43297},[525],[47,43299,43301,43312],{"className":43300,"style":36767},[529],[47,43302,43303,43306],{"style":17673},[47,43304],{"className":43305,"style":929},[537],[47,43307,43309],{"className":43308,"style":42975},[542,543,544,545],[47,43310,724],{"className":43311,"style":42975},[508,581,545],[47,43313,43314,43317],{"style":16891},[47,43315],{"className":43316,"style":929},[537],[47,43318,43320],{"className":43319,"style":42975},[542,543,544,545],[47,43321,1166],{"className":43322,"style":42975},[508,581,545],[47,43324,616],{"className":43325},[615],[47,43327,43329],{"className":43328},[525],[47,43330,43332],{"className":43331,"style":17703},[529],[47,43333],{},[47,43335,13303],{"className":43336,"style":42975},[508],[47,43338,43340,43343],{"className":43339,"style":42975},[508],[47,43341,13164],{"className":43342,"style":42975},[508,581],[47,43344,43346],{"className":43345},[585],[47,43347,43349,43380],{"className":43348},[520,521],[47,43350,43352,43377],{"className":43351},[525],[47,43353,43355,43366],{"className":43354,"style":36767},[529],[47,43356,43357,43360],{"style":17673},[47,43358],{"className":43359,"style":929},[537],[47,43361,43363],{"className":43362,"style":42975},[542,543,544,545],[47,43364,724],{"className":43365,"style":42975},[508,581,545],[47,43367,43368,43371],{"style":16891},[47,43369],{"className":43370,"style":929},[537],[47,43372,43374],{"className":43373,"style":42975},[542,543,544,545],[47,43375,1166],{"className":43376,"style":42975},[508,581,545],[47,43378,616],{"className":43379},[615],[47,43381,43383],{"className":43382},[525],[47,43384,43386],{"className":43385,"style":17703},[529],[47,43387],{},[47,43389,1201],{"className":43390,"style":42975},[702],[47,43392,43393,43396],{"style":663},[47,43394],{"className":43395,"style":538},[537],[47,43397],{"className":43398,"style":43399},[670],"color:red;border-bottom-width:0.04em;",[47,43401,43402,43405],{"style":16247},[47,43403],{"className":43404,"style":538},[537],[47,43406,43408,43451,43454,43505,43508,43559],{"className":43407,"style":42975},[508],[47,43409,43411,43414],{"className":43410,"style":42975},[508],[47,43412,5974],{"className":43413,"style":43213},[508,581],[47,43415,43417],{"className":43416},[585],[47,43418,43420,43443],{"className":43419},[520,521],[47,43421,43423,43440],{"className":43422},[525],[47,43424,43426],{"className":43425,"style":1635},[529],[47,43427,43428,43431],{"style":7830},[47,43429],{"className":43430,"style":929},[537],[47,43432,43434],{"className":43433,"style":42975},[542,543,544,545],[47,43435,43437],{"className":43436,"style":42975},[508,545],[47,43438,14413],{"className":43439,"style":43008},[508,581,545],[47,43441,616],{"className":43442},[615],[47,43444,43446],{"className":43445},[525],[47,43447,43449],{"className":43448,"style":1657},[529],[47,43450],{},[47,43452,1187],{"className":43453,"style":42975},[512],[47,43455,43457,43460],{"className":43456,"style":42975},[508],[47,43458,285],{"className":43459,"style":42975},[508,581],[47,43461,43463],{"className":43462},[585],[47,43464,43466,43497],{"className":43465},[520,521],[47,43467,43469,43494],{"className":43468},[525],[47,43470,43472,43483],{"className":43471,"style":19211},[529],[47,43473,43474,43477],{"style":17673},[47,43475],{"className":43476,"style":929},[537],[47,43478,43480],{"className":43479,"style":42975},[542,543,544,545],[47,43481,724],{"className":43482,"style":42975},[508,581,545],[47,43484,43485,43488],{"style":925},[47,43486],{"className":43487,"style":929},[537],[47,43489,43491],{"className":43490,"style":42975},[542,543,544,545],[47,43492,1166],{"className":43493,"style":42975},[508,581,545],[47,43495,616],{"className":43496},[615],[47,43498,43500],{"className":43499},[525],[47,43501,43503],{"className":43502,"style":17703},[529],[47,43504],{},[47,43506,13303],{"className":43507,"style":42975},[508],[47,43509,43511,43514],{"className":43510,"style":42975},[508],[47,43512,13164],{"className":43513,"style":42975},[508,581],[47,43515,43517],{"className":43516},[585],[47,43518,43520,43551],{"className":43519},[520,521],[47,43521,43523,43548],{"className":43522},[525],[47,43524,43526,43537],{"className":43525,"style":19211},[529],[47,43527,43528,43531],{"style":17673},[47,43529],{"className":43530,"style":929},[537],[47,43532,43534],{"className":43533,"style":42975},[542,543,544,545],[47,43535,724],{"className":43536,"style":42975},[508,581,545],[47,43538,43539,43542],{"style":925},[47,43540],{"className":43541,"style":929},[537],[47,43543,43545],{"className":43544,"style":42975},[542,543,544,545],[47,43546,1166],{"className":43547,"style":42975},[508,581,545],[47,43549,616],{"className":43550},[615],[47,43552,43554],{"className":43553},[525],[47,43555,43557],{"className":43556,"style":17703},[529],[47,43558],{},[47,43560,1201],{"className":43561,"style":42975},[702],[47,43563,616],{"className":43564},[615],[47,43566,43568],{"className":43567},[525],[47,43569,43571],{"className":43570,"style":16312},[529],[47,43572],{},[47,43574],{"className":43575},[702,513],[47,43577,1192],{"className":43578},[1299],[47,43580],{"className":43581,"style":1303},[888],[47,43583],{"className":43584,"style":1303},[888],[47,43586,43589],{"className":43587,"style":43588},[508],"color:skyblue;",[47,43590,43592],{"className":43591,"style":43588},[508,1342],"clip",[47,43594],{"className":43595,"style":1303},[888],[47,43597,43599,43608,44004,44007,44011,44014,44017,44021,44024,44027,44030,44033,44036,44039,44042,44045,44048,44051,44054],{"className":43598,"style":43588},[1349],[47,43600,43603],{"className":43601,"style":43602},[512,1353],"color:skyblue;top:0em;",[47,43604,43606],{"className":43605,"style":43588},[1358,544],[47,43607,1187],{"style":43588},[47,43609,43611,43614,44001],{"className":43610,"style":43588},[508],[47,43612],{"className":43613},[512,513],[47,43615,43617],{"className":43616},[467],[47,43618,43620,43993],{"className":43619},[520,521],[47,43621,43623,43990],{"className":43622},[525],[47,43624,43626,43819,43828],{"className":43625,"style":36665},[529],[47,43627,43628,43631],{"style":16181},[47,43629],{"className":43630,"style":538},[537],[47,43632,43634,43708,43711,43762,43765,43816],{"className":43633,"style":43588},[508],[47,43635,43637,43641],{"className":43636,"style":43588},[508],[47,43638,5974],{"className":43639,"style":43640},[508,581],"margin-right:0.13889em;color:skyblue;",[47,43642,43644],{"className":43643},[585],[47,43645,43647,43700],{"className":43646},[520,521],[47,43648,43650,43697],{"className":43649},[525],[47,43651,43653],{"className":43652,"style":1635},[529],[47,43654,43655,43658],{"style":7830},[47,43656],{"className":43657,"style":929},[537],[47,43659,43661],{"className":43660,"style":43588},[542,543,544,545],[47,43662,43664],{"className":43663,"style":43588},[508,545],[47,43665,43667,43671],{"className":43666,"style":43588},[508,545],[47,43668,14413],{"className":43669,"style":43670},[508,581,545],"margin-right:0.02778em;color:skyblue;",[47,43672,43674],{"className":43673},[585],[47,43675,43677],{"className":43676},[520],[47,43678,43680],{"className":43679},[525],[47,43681,43683],{"className":43682,"style":19506},[529],[47,43684,43685,43688],{"style":19509},[47,43686],{"className":43687,"style":602},[537],[47,43689,43691],{"className":43690,"style":43588},[542,606,607,545],[47,43692,43694],{"className":43693,"style":43588},[508,545],[47,43695,19346],{"className":43696,"style":43588},[508,545],[47,43698,616],{"className":43699},[615],[47,43701,43703],{"className":43702},[525],[47,43704,43706],{"className":43705,"style":1657},[529],[47,43707],{},[47,43709,1187],{"className":43710,"style":43588},[512],[47,43712,43714,43717],{"className":43713,"style":43588},[508],[47,43715,285],{"className":43716,"style":43588},[508,581],[47,43718,43720],{"className":43719},[585],[47,43721,43723,43754],{"className":43722},[520,521],[47,43724,43726,43751],{"className":43725},[525],[47,43727,43729,43740],{"className":43728,"style":36767},[529],[47,43730,43731,43734],{"style":17673},[47,43732],{"className":43733,"style":929},[537],[47,43735,43737],{"className":43736,"style":43588},[542,543,544,545],[47,43738,724],{"className":43739,"style":43588},[508,581,545],[47,43741,43742,43745],{"style":16891},[47,43743],{"className":43744,"style":929},[537],[47,43746,43748],{"className":43747,"style":43588},[542,543,544,545],[47,43749,1166],{"className":43750,"style":43588},[508,581,545],[47,43752,616],{"className":43753},[615],[47,43755,43757],{"className":43756},[525],[47,43758,43760],{"className":43759,"style":17703},[529],[47,43761],{},[47,43763,13303],{"className":43764,"style":43588},[508],[47,43766,43768,43771],{"className":43767,"style":43588},[508],[47,43769,13164],{"className":43770,"style":43588},[508,581],[47,43772,43774],{"className":43773},[585],[47,43775,43777,43808],{"className":43776},[520,521],[47,43778,43780,43805],{"className":43779},[525],[47,43781,43783,43794],{"className":43782,"style":36767},[529],[47,43784,43785,43788],{"style":17673},[47,43786],{"className":43787,"style":929},[537],[47,43789,43791],{"className":43790,"style":43588},[542,543,544,545],[47,43792,724],{"className":43793,"style":43588},[508,581,545],[47,43795,43796,43799],{"style":16891},[47,43797],{"className":43798,"style":929},[537],[47,43800,43802],{"className":43801,"style":43588},[542,543,544,545],[47,43803,1166],{"className":43804,"style":43588},[508,581,545],[47,43806,616],{"className":43807},[615],[47,43809,43811],{"className":43810},[525],[47,43812,43814],{"className":43813,"style":17703},[529],[47,43815],{},[47,43817,1201],{"className":43818,"style":43588},[702],[47,43820,43821,43824],{"style":663},[47,43822],{"className":43823,"style":538},[537],[47,43825],{"className":43826,"style":43827},[670],"color:skyblue;border-bottom-width:0.04em;",[47,43829,43830,43833],{"style":16247},[47,43831],{"className":43832,"style":538},[537],[47,43834,43836,43879,43882,43933,43936,43987],{"className":43835,"style":43588},[508],[47,43837,43839,43842],{"className":43838,"style":43588},[508],[47,43840,5974],{"className":43841,"style":43640},[508,581],[47,43843,43845],{"className":43844},[585],[47,43846,43848,43871],{"className":43847},[520,521],[47,43849,43851,43868],{"className":43850},[525],[47,43852,43854],{"className":43853,"style":1635},[529],[47,43855,43856,43859],{"style":7830},[47,43857],{"className":43858,"style":929},[537],[47,43860,43862],{"className":43861,"style":43588},[542,543,544,545],[47,43863,43865],{"className":43864,"style":43588},[508,545],[47,43866,14413],{"className":43867,"style":43670},[508,581,545],[47,43869,616],{"className":43870},[615],[47,43872,43874],{"className":43873},[525],[47,43875,43877],{"className":43876,"style":1657},[529],[47,43878],{},[47,43880,1187],{"className":43881,"style":43588},[512],[47,43883,43885,43888],{"className":43884,"style":43588},[508],[47,43886,285],{"className":43887,"style":43588},[508,581],[47,43889,43891],{"className":43890},[585],[47,43892,43894,43925],{"className":43893},[520,521],[47,43895,43897,43922],{"className":43896},[525],[47,43898,43900,43911],{"className":43899,"style":19211},[529],[47,43901,43902,43905],{"style":17673},[47,43903],{"className":43904,"style":929},[537],[47,43906,43908],{"className":43907,"style":43588},[542,543,544,545],[47,43909,724],{"className":43910,"style":43588},[508,581,545],[47,43912,43913,43916],{"style":925},[47,43914],{"className":43915,"style":929},[537],[47,43917,43919],{"className":43918,"style":43588},[542,543,544,545],[47,43920,1166],{"className":43921,"style":43588},[508,581,545],[47,43923,616],{"className":43924},[615],[47,43926,43928],{"className":43927},[525],[47,43929,43931],{"className":43930,"style":17703},[529],[47,43932],{},[47,43934,13303],{"className":43935,"style":43588},[508],[47,43937,43939,43942],{"className":43938,"style":43588},[508],[47,43940,13164],{"className":43941,"style":43588},[508,581],[47,43943,43945],{"className":43944},[585],[47,43946,43948,43979],{"className":43947},[520,521],[47,43949,43951,43976],{"className":43950},[525],[47,43952,43954,43965],{"className":43953,"style":19211},[529],[47,43955,43956,43959],{"style":17673},[47,43957],{"className":43958,"style":929},[537],[47,43960,43962],{"className":43961,"style":43588},[542,543,544,545],[47,43963,724],{"className":43964,"style":43588},[508,581,545],[47,43966,43967,43970],{"style":925},[47,43968],{"className":43969,"style":929},[537],[47,43971,43973],{"className":43972,"style":43588},[542,543,544,545],[47,43974,1166],{"className":43975,"style":43588},[508,581,545],[47,43977,616],{"className":43978},[615],[47,43980,43982],{"className":43981},[525],[47,43983,43985],{"className":43984,"style":17703},[529],[47,43986],{},[47,43988,1201],{"className":43989,"style":43588},[702],[47,43991,616],{"className":43992},[615],[47,43994,43996],{"className":43995},[525],[47,43997,43999],{"className":43998,"style":16312},[529],[47,44000],{},[47,44002],{"className":44003},[702,513],[47,44005,1192],{"className":44006,"style":43588},[1299],[47,44008],{"className":44009,"style":44010},[888],"color:skyblue;margin-right:0.1667em;",[47,44012],{"className":44013,"style":44010},[888],[47,44015,472],{"className":44016,"style":43588},[508],[47,44018],{"className":44019,"style":44020},[888],"color:skyblue;margin-right:0.2222em;",[47,44022,18727],{"className":44023,"style":43588},[942],[47,44025],{"className":44026,"style":44020},[888],[47,44028,42207],{"className":44029,"style":43588},[508,581],[47,44031,1192],{"className":44032,"style":43588},[1299],[47,44034],{"className":44035,"style":44010},[888],[47,44037],{"className":44038,"style":44010},[888],[47,44040,472],{"className":44041,"style":43588},[508],[47,44043],{"className":44044,"style":44020},[888],[47,44046,6147],{"className":44047,"style":43588},[942],[47,44049],{"className":44050,"style":44020},[888],[47,44052,42207],{"className":44053,"style":43588},[508,581],[47,44055,44057],{"className":44056,"style":43602},[702,1353],[47,44058,44060],{"className":44059,"style":43588},[1358,544],[47,44061,1201],{"style":43588},[47,44063],{"className":44064,"style":1303},[888],[47,44066,44068,44071],{"className":44067,"style":43588},[508],[47,44069,1163],{"className":44070,"style":43588},[508,581],[47,44072,44074],{"className":44073},[585],[47,44075,44077,44150],{"className":44076},[520,521],[47,44078,44080,44147],{"className":44079},[525],[47,44081,44083,44126],{"className":44082,"style":25158},[529],[47,44084,44085,44088],{"style":17673},[47,44086],{"className":44087,"style":929},[537],[47,44089,44091],{"className":44090,"style":43588},[542,543,544,545],[47,44092,44094],{"className":44093,"style":43588},[508,545],[47,44095,44097,44100],{"className":44096,"style":43588},[508,545],[47,44098,14413],{"className":44099,"style":43670},[508,581,545],[47,44101,44103],{"className":44102},[585],[47,44104,44106],{"className":44105},[520],[47,44107,44109],{"className":44108},[525],[47,44110,44112],{"className":44111,"style":19506},[529],[47,44113,44114,44117],{"style":19509},[47,44115],{"className":44116,"style":602},[537],[47,44118,44120],{"className":44119,"style":43588},[542,606,607,545],[47,44121,44123],{"className":44122,"style":43588},[508,545],[47,44124,19346],{"className":44125,"style":43588},[508,545],[47,44127,44128,44131],{"style":16788},[47,44129],{"className":44130,"style":929},[537],[47,44132,44134],{"className":44133,"style":43588},[542,543,544,545],[47,44135,44137,44140,44143],{"className":44136,"style":43588},[508,545],[47,44138,27872],{"className":44139,"style":43588},[508,581,545],[47,44141,1163],{"className":44142,"style":43588},[508,581,545],[47,44144,5977],{"className":44145,"style":44146},[508,581,545],"margin-right:0.05764em;color:skyblue;",[47,44148,616],{"className":44149},[615],[47,44151,44153],{"className":44152},[525],[47,44154,44156],{"className":44155,"style":17703},[529],[47,44157],{},[47,44159,1187],{"className":44160,"style":43588},[512],[47,44162,44164,44167],{"className":44163,"style":43588},[508],[47,44165,13164],{"className":44166,"style":43588},[508,581],[47,44168,44170],{"className":44169},[585],[47,44171,44173,44204],{"className":44172},[520,521],[47,44174,44176,44201],{"className":44175},[525],[47,44177,44179,44190],{"className":44178,"style":17670},[529],[47,44180,44181,44184],{"style":17673},[47,44182],{"className":44183,"style":929},[537],[47,44185,44187],{"className":44186,"style":43588},[542,543,544,545],[47,44188,724],{"className":44189,"style":43588},[508,581,545],[47,44191,44192,44195],{"style":16788},[47,44193],{"className":44194,"style":929},[537],[47,44196,44198],{"className":44197,"style":43588},[542,543,544,545],[47,44199,1166],{"className":44200,"style":43588},[508,581,545],[47,44202,616],{"className":44203},[615],[47,44205,44207],{"className":44206},[525],[47,44208,44210],{"className":44209,"style":17703},[529],[47,44211],{},[47,44213,1192],{"className":44214,"style":43588},[1299],[47,44216],{"className":44217,"style":1303},[888],[47,44219,44221,44224],{"className":44220,"style":43588},[508],[47,44222,285],{"className":44223,"style":43588},[508,581],[47,44225,44227],{"className":44226},[585],[47,44228,44230,44261],{"className":44229},[520,521],[47,44231,44233,44258],{"className":44232},[525],[47,44234,44236,44247],{"className":44235,"style":17670},[529],[47,44237,44238,44241],{"style":17673},[47,44239],{"className":44240,"style":929},[537],[47,44242,44244],{"className":44243,"style":43588},[542,543,544,545],[47,44245,724],{"className":44246,"style":43588},[508,581,545],[47,44248,44249,44252],{"style":16788},[47,44250],{"className":44251,"style":929},[537],[47,44253,44255],{"className":44254,"style":43588},[542,543,544,545],[47,44256,1166],{"className":44257,"style":43588},[508,581,545],[47,44259,616],{"className":44260},[615],[47,44262,44264],{"className":44263},[525],[47,44265,44267],{"className":44266,"style":17703},[529],[47,44268],{},[47,44270,1201],{"className":44271,"style":43588},[702],[47,44273,44275],{"className":44274,"style":1354},[702,1353],[47,44276,1201],{"className":44277},[1358,544],[10,44279,44280],{"id":44280},"参考资料",[14,44282,44283],{},[285,44284,44287],{"href":44285,"rel":44286},"https://www.bilibili.com/video/BV1iz421h7gb/",[289],"零基础学习强化学习算法：ppo",{"title":42,"searchDepth":64,"depth":64,"links":44289},[44290,44291,44292,44293,44296,44300],{"id":13035,"depth":64,"text":13035},{"id":13041,"depth":64,"text":13041},{"id":14307,"depth":64,"text":14307},{"id":19823,"depth":64,"text":19824,"children":44294},[44295],{"id":27800,"depth":74,"text":27801},{"id":32122,"depth":64,"text":32123,"children":44297},[44298,44299],{"id":32129,"depth":74,"text":32129},{"id":33934,"depth":74,"text":33935},{"id":44280,"depth":64,"text":44280},"记一下学习 PPO中学习到的东西",{"date":44303,"image":44304,"alt":392,"tags":44305,"published":325},"8 Mar 2026","/blogs-img/blog2.jpg",[44306,13023],"Reinforcement learning","/blogs/ppo",{"title":13030,"description":44301},"blogs/2. PPO学习笔记","Sx3SA-gd0nzBc5-IShm_PpZrHX9EMT8A7rF2k0GLT9I",{"id":4,"title":5,"body":44312,"description":5,"extension":388,"meta":44568,"navigation":325,"ogImage":391,"path":396,"seo":44570,"stem":398,"__hash__":399},{"type":7,"value":44313,"toc":44551},[44314,44316,44318,44320,44322,44324,44326,44328,44330,44360,44362,44364,44366,44368,44420,44422,44432,44434,44436,44438,44440,44442,44458,44460,44462,44464,44480,44482,44486,44488,44490,44492,44497,44499,44501,44503,44505,44507,44545,44547,44549],[10,44315,12],{"id":12},[14,44317,16],{},[10,44319,20],{"id":19},[14,44321,23],{},[14,44323,26],{},[10,44325,29],{"id":29},[14,44327,32],{},[14,44329,35],{},[37,44331,44332],{"className":39,"code":40,"language":41,"meta":42,"style":42},[44,44333,44334,44342,44348],{"__ignoreMap":42},[47,44335,44336,44338,44340],{"class":49,"line":50},[47,44337,54],{"class":53},[47,44339,58],{"class":57},[47,44341,61],{"class":57},[47,44343,44344,44346],{"class":49,"line":64},[47,44345,68],{"class":67},[47,44347,71],{"class":57},[47,44349,44350,44352,44354,44356,44358],{"class":49,"line":74},[47,44351,78],{"class":77},[47,44353,82],{"class":81},[47,44355,85],{"class":57},[47,44357,88],{"class":53},[47,44359,91],{"class":57},[14,44361,94],{},[10,44363,98],{"id":97},[14,44365,101],{},[14,44367,104],{},[37,44369,44370],{"className":39,"code":107,"language":41,"meta":42,"style":42},[44,44371,44372,44406],{"__ignoreMap":42},[47,44373,44374,44376,44378,44380,44382,44384,44386,44388,44390,44392,44394,44396,44398,44400,44402,44404],{"class":49,"line":50},[47,44375,54],{"class":53},[47,44377,58],{"class":57},[47,44379,119],{"class":118},[47,44381,122],{"class":57},[47,44383,126],{"class":125},[47,44385,129],{"class":57},[47,44387,132],{"class":77},[47,44389,135],{"class":81},[47,44391,138],{"class":77},[47,44393,141],{"class":81},[47,44395,144],{"class":57},[47,44397,29],{"class":77},[47,44399,141],{"class":81},[47,44401,151],{"class":77},[47,44403,154],{"class":57},[47,44405,157],{"class":125},[47,44407,44408,44410,44412,44414,44416,44418],{"class":49,"line":64},[47,44409,162],{"class":67},[47,44411,165],{"class":125},[47,44413,168],{"class":57},[47,44415,171],{"class":125},[47,44417,174],{"class":81},[47,44419,177],{"class":57},[14,44421,180],{},[14,44423,183,44424,187,44426,191,44428,195,44430,199],{},[44,44425,186],{},[44,44427,190],{},[44,44429,194],{},[44,44431,198],{},[10,44433,202],{"id":202},[14,44435,205],{},[207,44437,209],{"id":209},[14,44439,212],{},[14,44441,215],{},[37,44443,44444],{"className":39,"code":218,"language":41,"meta":42,"style":42},[44,44445,44446],{"__ignoreMap":42},[47,44447,44448,44450,44452,44454,44456],{"class":49,"line":50},[47,44449,54],{"class":53},[47,44451,58],{"class":57},[47,44453,229],{"class":57},[47,44455,232],{"class":77},[47,44457,235],{"class":57},[207,44459,238],{"id":238},[14,44461,241],{},[14,44463,215],{},[37,44465,44466],{"className":39,"code":246,"language":41,"meta":42,"style":42},[44,44467,44468],{"__ignoreMap":42},[47,44469,44470,44472,44474,44476,44478],{"class":49,"line":50},[47,44471,54],{"class":53},[47,44473,58],{"class":57},[47,44475,257],{"class":57},[47,44477,232],{"class":77},[47,44479,262],{"class":57},[207,44481,54],{"id":54},[14,44483,267,44484,271],{},[44,44485,270],{},[10,44487,274],{"id":274},[207,44489,277],{"id":277},[14,44491,280],{},[14,44493,283,44494,290],{},[285,44495,287],{"href":287,"rel":44496},[289],[207,44498,293],{"id":293},[14,44500,296],{},[207,44502,299],{"id":299},[14,44504,302],{},[14,44506,305],{},[37,44508,44509],{"className":39,"code":308,"language":41,"meta":42,"style":42},[44,44510,44511,44519,44523,44527,44533],{"__ignoreMap":42},[47,44512,44513,44515,44517],{"class":49,"line":50},[47,44514,68],{"class":67},[47,44516,317],{"class":77},[47,44518,320],{"class":57},[47,44520,44521],{"class":49,"line":64},[47,44522,326],{"emptyLinePlaceholder":325},[47,44524,44525],{"class":49,"line":74},[47,44526,332],{"class":331},[47,44528,44529,44531],{"class":49,"line":335},[47,44530,338],{"class":53},[47,44532,341],{"class":57},[47,44534,44535,44537,44539,44541,44543],{"class":49,"line":344},[47,44536,347],{"class":53},[47,44538,350],{"class":57},[47,44540,354],{"class":353},[47,44542,357],{"class":53},[47,44544,360],{"class":57},[207,44546,363],{"id":363},[14,44548,366],{},[368,44550,370],{},{"title":42,"searchDepth":64,"depth":64,"links":44552},[44553,44554,44555,44556,44557,44562],{"id":12,"depth":64,"text":12},{"id":19,"depth":64,"text":20},{"id":29,"depth":64,"text":29},{"id":97,"depth":64,"text":98},{"id":202,"depth":64,"text":202,"children":44558},[44559,44560,44561],{"id":209,"depth":74,"text":209},{"id":238,"depth":74,"text":238},{"id":54,"depth":74,"text":54},{"id":274,"depth":64,"text":274,"children":44563},[44564,44565,44566,44567],{"id":277,"depth":74,"text":277},{"id":293,"depth":74,"text":293},{"id":299,"depth":74,"text":299},{"id":363,"depth":74,"text":363},{"date":390,"image":391,"alt":392,"tags":44569,"published":325},[394,395],{"title":5,"description":5},1777363047842]