[1090] | 1 | V t� kapitole je pops�jednoduch��zkouman�ite{astrom1986dual}. Na n�jsou porovn� ��lgoritmy uveden� p�l�apitole. |
---|
[918] | 2 | |
---|
[891] | 3 | \section{Popis syst�} |
---|
[918] | 4 | V�syst� je pops�jako |
---|
| 5 | \begin{gather} |
---|
| 6 | \label{simple} |
---|
[1090] | 7 | y_{t+1}=y_t+\theta u_t+v_{t+1} \qquad t=0,\ldots,N-1,\\ |
---|
| 8 | v_{t+1}\sim N(0,\sigma^2), |
---|
[918] | 9 | \end{gather} |
---|
[1090] | 10 | kde rozptyl �umu $\sigma$ je zn� |
---|
[918] | 11 | |
---|
[1090] | 12 | O nezn�m parametru $\theta$ m� v �e $t$ informaci v podob�ostate� statistiky $T_t=(\hat{\theta},P_t)$, tvo�st� hodnotou a rozptylem. P�kl�me nekorelovanost $\theta$ s �umem, tedy �e |
---|
| 13 | \begin{equation} |
---|
| 14 | \cov(v_{t+1},\theta)=0. |
---|
| 15 | \end{equation} |
---|
| 16 | |
---|
[918] | 17 | Ztr�vou funkci vol� kvadratickou, tedy |
---|
| 18 | \begin{equation} |
---|
[1090] | 19 | g(y_{0:N},u_{0:N-1})=\sum_{t=0}^{N-1}y_{t+1}^2. |
---|
[918] | 20 | \end{equation} |
---|
| 21 | |
---|
[930] | 22 | Odhadovac�rocedurou pro parametr $\theta$ je Kalman�ltr. Pro syst�\eqref{simple} m�var |
---|
[918] | 23 | \begin{gather} |
---|
| 24 | \label{kal} |
---|
| 25 | K_t=\frac{u_tP_t}{u_t^2P_t+\sigma^2}\\ |
---|
| 26 | \hat{\theta}_{t+1}=\hat{\theta}_t+K_t(y_{t+1}-u_t\hat{\theta}_t),\\ |
---|
| 27 | P_{t+1}=(1-K_tu_t)P_t. |
---|
| 28 | \end{gather} |
---|
| 29 | |
---|
[1090] | 30 | Hyperstav syst� $H_t$ tvo�ktor $(y_t,\hat{\theta}_t,P_t)$. O��n�tr� je |
---|
[918] | 31 | \begin{equation} |
---|
[1090] | 32 | J_t(H_t)=\min_{u_t \in U_t}\E_{y_{t+1},v_t}\left\{y_{t+1}^2+J_{t+1}(H_{t+1})|H_t,u_t\right\}, \qquad t=0,\ldots,N-1. |
---|
[918] | 33 | \end{equation} |
---|
| 34 | |
---|
| 35 | Ta po dosazen� \eqref{simple} a �te�m proveden�t� hodnoty p� na tvar |
---|
| 36 | \begin{gather} |
---|
| 37 | \label{dos} |
---|
| 38 | J_t(y_t,\theta_t)=\min_{u_t \in U_t}\left\{(y_t+\hat{\theta}_tu_t)^2+u_t^2P_t+\sigma^2+\E_{y_{t+1},v_t}(J_{t+1}(y_{t+1},\theta_{t+1}))|y_t,\theta_t,u_t\right\}. |
---|
| 39 | \end{gather} |
---|
| 40 | |
---|
| 41 | \section{Specifika jednotliv��up� tomto odd� jsou pops� n�er�spekty algoritm�er�udeme srovn�t, p�likaci na syst�\eqref{simple}. |
---|
| 42 | |
---|
| 43 | \subsection{Certainty equivalent control} |
---|
| 44 | O��n�tr� \eqref{CE} prejde v |
---|
| 45 | \begin{gather} |
---|
| 46 | J_t(H_t)=\min_{u_t \in U_t}\left\{\hat{y}_t^2 +J_{t+1}(\hat{H}_{t+1})|I_t,\theta_t,u_t\right\}. |
---|
[1090] | 47 | \end{gather} |
---|
[918] | 48 | St� hodnota v� je |
---|
| 49 | \begin{equation} |
---|
| 50 | \hat{y}_{t+1}=y_t+\hat{\theta}_tu_t |
---|
| 51 | \end{equation} |
---|
| 52 | a rozhodnut�ude tedy |
---|
| 53 | \begin{equation} |
---|
| 54 | \mu_t(y_t,\hat{\theta}_t)=-\frac{y_t}{\hat{\theta}_t}. |
---|
| 55 | \end{equation} |
---|
| 56 | |
---|
| 57 | \subsection{Metoda separace} |
---|
| 58 | V prvn�� metody separace polo�� ���h |
---|
| 59 | \begin{equation} |
---|
| 60 | u_0=\sqrt{C-\frac{1}{P_0}}. |
---|
| 61 | \end{equation} |
---|
| 62 | T�se dle \eqref{kal} sn� rozptyl $P_0$ nezn�ho parametru $\theta$ na $\frac{1}{C}$. Konstanta $C$ by m� b�ena dostate� mal�aby odhad $\hat{\theta}$ pro druhou f� ��yl dostate� bl�o skute� hodnot�arametru $\theta$. P�ovn� jednotliv�goritm�l�me $C=100$. |
---|
| 63 | |
---|
| 64 | \subsection{SIDP} |
---|
| 65 | Dle \eqref{dos} je optim��u_t$ z�sl�a $(y_t,\hat{\theta}_t,P_t)$. P�mulaci m� tedy v ka�d��ov�okam�iku $t$ diskretizovat t�enzion��rostor nez�sle prom��le \cite{astrom1986dual} je v�ak p�amotnou simulac�hodn�� k transformaci prostoru $(y_t,\hat{\theta}_t,P_t,u_t)$ do nov�om��\eta_t,\beta_t,\zeta_t,\nu_t)$ dle |
---|
[919] | 66 | \begin{gather} |
---|
[918] | 67 | \eta_t=\frac{y_t}{\sigma} \\ |
---|
| 68 | \beta_t=\frac{\hat{\theta}_t}{\sqrt{P_t}} \\ |
---|
| 69 | \zeta_t=\frac{1}{\sqrt{P_t}} \\ |
---|
| 70 | \nu_t=\frac{u_t\sqrt{P_t}}{\sigma} |
---|
| 71 | \end{gather} |
---|
| 72 | |
---|
| 73 | Sou�n�� neur�ost ve v� \eqref{simple} reprezentovat jedinou normalizovanou n�dnou veli�ou podle |
---|
| 74 | \begin{equation} |
---|
| 75 | s_t=\frac{y_{t+1}-y_t+\hat{\theta}_tu_t}{\sqrt{u_t^2P_t+\sigma^2}} \sim N(0,1). |
---|
| 76 | \end{equation} |
---|
| 77 | |
---|
| 78 | Rovnice pro v�\eqref{simple} a n�eduj� odhad nezn�ho parametru \eqref{kal} tak p� v |
---|
| 79 | \begin{gather} |
---|
| 80 | \eta_{t+1}=\eta_t+\beta_t\nu_t+\sqrt{1+\nu^2}s_t\\ |
---|
| 81 | \beta_{t+1}=\sqrt{1+\nu^2}\beta_t+\nu_ts_t |
---|
| 82 | \end{gather} |
---|
| 83 | |
---|
| 84 | P�me-li k vhodn�praven���n�tr�, dostaneme |
---|
| 85 | \begin{align} |
---|
| 86 | V_t(\eta_t,\beta_t,\zeta_t)&=\frac{J_t(y_t,\hat{\theta}_t,P_t)}{\sigma^2}\\ |
---|
| 87 | &=\min_{\nu_t }\left\{(\eta_t+\beta_t\nu_t)^2+\nu_t^2+1+\E_{y_{t+1},v_t}(V_{t+1}(\eta_{t+1},\beta_{t+1},\zeta))\right\}. |
---|
| 88 | \end{align} |
---|
| 89 | |
---|
| 90 | Nyn�po�me o��nou ztr� pro $N-1$. |
---|
| 91 | \begin{equation} |
---|
| 92 | V_{N-1}(\eta_{N-1},\beta_{N-1},\zeta_{N-1})=\min_{\nu_{N-1}}\left\{(\eta_{N-1}+\beta_{N-1}\nu_{N-1})^2+\nu_{N-1}^2+1\right\}. |
---|
| 93 | \end{equation} |
---|
| 94 | |
---|
| 95 | Derivac��� optim���h jako |
---|
| 96 | \begin{equation} |
---|
| 97 | \label{optcon} |
---|
| 98 | \nu_{N-1}=-\frac{\eta_{N-1}\beta_{N-1}}{1+\beta_{N-1}^2} |
---|
| 99 | \end{equation} |
---|
| 100 | a o��nou ztr� |
---|
| 101 | \begin{equation} |
---|
| 102 | V_{N-1}(\eta_{N-1},\beta_{N-1},\zeta_{N-1})= \frac{\eta_{N-1}^2+1}{\beta_{N-1}^2+1} |
---|
| 103 | \end{equation} |
---|
| 104 | |
---|
| 105 | Proto�e optim���h $\nu_{N-1}$ ani o��n�tr� $V_{N-1}$ nez�s�a $\zeta_{N-1}$, d� tvaru $V_t$ nebude rovn�optim���h $\nu_t$ a o��n�tr� $V_t$ z�set na $\zeta_t$. P�skretizaci tedy sta�uva�ovat pouze dvoudimenzion��rostor nez�sle prom��\eta_t,\beta_t)$. |
---|
| 106 | |
---|
| 107 | \section{Srovn� jednotliv��up� t� sekci jsou porovn� popsan��c�lgoritmy na syst� \eqref{simple}. |
---|
[930] | 108 | POPIS EXPERIMENTU |
---|