-Policy函数拟合算法-Policy Gradient算法02-2：目标函数/metrics的选取02【average one-step reward】)
2、average one-step reward2种常见表达形式：rˉπ≐∑s∈Sdπ(s)rπ(s)≐limn→∞1nE[∑k=1nRt+k]\begin{aligned} \bar{r}_{\pi}\doteq\sum_{s\in\mathcal{S}}d_{\pi}\left(s\right)r_{\pi}\left(s\right) \\[4ex] \doteq\lim_{n\to\infty}\frac{1}{n}\mathbb{E}\left[\sum_{k=1}^{n}R_{t+k}\right] \\[4ex] \end{aligned}rˉπ≐s∈S∑dπ(s)rπ(s)≐n→∞limn1E[k=1∑nRt+k]第二大类是average one-step reward, 或者简称为 average reward.具体地′_{\prime}′the metrics是rˉπ≐∑s∈Sdπ(s)rπ(s)=E[rπ(S)]\color{red}{ \bar{r}_{\pi}\doteq\sum_{s\in\mathcal{S}}d_{\pi}\left(s\right)r_{\pi}\left(s\right)=\mathbb{E}[r_{\pi}\left(S\right)]}rˉπ≐ RL-赵-(九)-Policy函数拟合算法-Policy Gradient算法02-2：目标函数/metrics的选取02【average one-step reward】
2、average one-step reward2种常见表达形式：rˉπ≐∑s∈Sdπ(s)rπ(s)≐limn→∞1nE[∑k=1nRt+k]\begin{aligned} \bar{r}_{\pi}\doteq\sum_{s\in\mathcal{S}}d_{\pi}\left(s\right)r_{\pi}\left(s\right) \\[4ex] \doteq\lim_{n\to\infty}\frac{1}{n}\mathbb{E}\left[\sum_{k=1}^{n}R_{t+k}\right] \\[4ex] \end{aligned}rˉπ≐s∈S∑dπ(s)rπ(s)≐n→∞limn1E[k=1∑nRt+k]第二大类是average one-step reward, 或者简称为 average reward.具体地′_{\prime}′the metrics是rˉπ≐∑s∈Sdπ(s)rπ(s)=E[rπ(S)]\color{red}{ \bar{r}_{\pi}\doteq\sum_{s\in\mathcal{S}}d_{\pi}\left(s\right)r_{\pi}\left(s\right)=\mathbb{E}[r_{\pi}\left(S\right)]}rˉπ≐