Changeset 323 for doc/html/tut_arx.html

Show
Ignore:
Timestamp:
04/23/09 21:12:23 (16 years ago)
Author:
smidl
Message:

doc

Files:
1 modified

Legend:

Unmodified
Added
Removed

  • doc/html/tut_arx.html

    r312 r323  
    6363<h1><a class="anchor" name="tut_arx">Theory of ARX model estimation </a></h1><p> 
    6464The <code>ARX</code> (AutoregRessive with eXogeneous input) model is defined as follows: <p class="formulaDsp"> 
    65 <img class="formulaDsp" alt="\[ y_t = \theta' \psi_t + \rho e_t \]" src="form_99.png"> 
     65<img class="formulaDsp" alt="\[ y_t = \theta' \psi_t + \rho e_t \]" src="form_107.png"> 
    6666<p> 
    67  where <img class="formulaInl" alt="$y_t$" src="form_100.png"> is the system output, <img class="formulaInl" alt="$[\theta,\rho]$" src="form_101.png"> is vector of unknown parameters, <img class="formulaInl" alt="$\psi_t$" src="form_102.png"> is an vector of data-dependent regressors, and noise <img class="formulaInl" alt="$e_t$" src="form_4.png"> is assumed to be Normal distributed <img class="formulaInl" alt="$\mathcal{N}(0,1)$" src="form_103.png">.<p> 
     67 where <img class="formulaInl" alt="$y_t$" src="form_42.png"> is the system output, <img class="formulaInl" alt="$[\theta,\rho]$" src="form_108.png"> is vector of unknown parameters, <img class="formulaInl" alt="$\psi_t$" src="form_109.png"> is an vector of data-dependent regressors, and noise <img class="formulaInl" alt="$e_t$" src="form_6.png"> is assumed to be Normal distributed <img class="formulaInl" alt="$\mathcal{N}(0,1)$" src="form_110.png">.<p> 
    6868Special cases include: <ul> 
    6969<li>estimation of unknown mean and variance of a Gaussian density from independent samples.</li> 
    … …  
    7575<dt>Information matrix </dt> 
    7676<dd>which is a sum of outer products <p class="formulaDsp"> 
    77 <img class="formulaDsp" alt="\[ V_t = \sum_{i=0}^{n} \left[\begin{array}{c}y_{t}\\ \psi_{t}\end{array}\right] \begin{array}{c} [y_{t}',\,\psi_{t}']\\ \\\end{array} \]" src="form_104.png"> 
     77<img class="formulaDsp" alt="\[ V_t = \sum_{i=0}^{n} \left[\begin{array}{c}y_{t}\\ \psi_{t}\end{array}\right] \begin{array}{c} [y_{t}',\,\psi_{t}']\\ \\\end{array} \]" src="form_111.png"> 
    7878<p> 
    7979 </dd> 
    8080<dt>"Degree of freedom" </dt> 
    8181<dd>which is an accumulator of number of data records <p class="formulaDsp"> 
    82 <img class="formulaDsp" alt="\[ \nu_t = \sum_{i=0}^{n} 1 \]" src="form_105.png"> 
     82<img class="formulaDsp" alt="\[ \nu_t = \sum_{i=0}^{n} 1 \]" src="form_112.png"> 
    8383<p> 
    8484 </dd> 
    … …  
    8787On-line estimation</a></h2> 
    8888For online estimation with stationary parameters can be easily achieved by collecting the sufficient statistics described above recursively.<p> 
    89 Extension to non-stationaly parameters, <img class="formulaInl" alt="$ \theta_t , r_t $" src="form_106.png"> can be achieved by operation called forgetting. This is an approximation of Bayesian filtering see [Kulhavy]. The resulting algorithm is defined by manipulation of sufficient statistics: <dl> 
     89Extension to non-stationaly parameters, <img class="formulaInl" alt="$ \theta_t , r_t $" src="form_113.png"> can be achieved by operation called forgetting. This is an approximation of Bayesian filtering see [Kulhavy]. The resulting algorithm is defined by manipulation of sufficient statistics: <dl> 
    9090<dt>Information matrix </dt> 
    9191<dd>which is a sum of outer products <p class="formulaDsp"> 
    92 <img class="formulaDsp" alt="\[ V_t = \phi V_{t-1} + \left[\begin{array}{c}y_{t}\\ \psi_{t}\end{array}\right] \begin{array}{c} [y_{t}',\,\psi_{t}']\\ \\\end{array} +(1-\phi) V_0 \]" src="form_121.png"> 
     92<img class="formulaDsp" alt="\[ V_t = \phi V_{t-1} + \left[\begin{array}{c}y_{t}\\ \psi_{t}\end{array}\right] \begin{array}{c} [y_{t}',\,\psi_{t}']\\ \\\end{array} +(1-\phi) V_0 \]" src="form_114.png"> 
    9393<p> 
    9494 </dd> 
    9595<dt>"Degree of freedom" </dt> 
    9696<dd>which is an accumulator of number of data records <p class="formulaDsp"> 
    97 <img class="formulaDsp" alt="\[ \nu_t = \phi \nu_{t-1} + 1 + (1-\phi) \nu_0 \]" src="form_122.png"> 
     97<img class="formulaDsp" alt="\[ \nu_t = \phi \nu_{t-1} + 1 + (1-\phi) \nu_0 \]" src="form_115.png"> 
    9898<p> 
    9999 </dd> 
    100100</dl> 
    101 where <img class="formulaInl" alt="$ \phi $" src="form_109.png"> is the forgetting factor, typically <img class="formulaInl" alt="$ \phi \in [0,1]$" src="form_110.png"> roughly corresponding to the effective length of the exponential window by relation:<p class="formulaDsp"> 
    102 <img class="formulaDsp" alt="\[ \mathrm{win_length} = \frac{1}{1-\phi}\]" src="form_111.png"> 
     101where <img class="formulaInl" alt="$ \phi $" src="form_116.png"> is the forgetting factor, typically <img class="formulaInl" alt="$ \phi \in [0,1]$" src="form_117.png"> roughly corresponding to the effective length of the exponential window by relation:<p class="formulaDsp"> 
     102<img class="formulaDsp" alt="\[ \mathrm{win_length} = \frac{1}{1-\phi}\]" src="form_118.png"> 
    103103<p> 
    104  Hence, <img class="formulaInl" alt="$ \phi=0.9 $" src="form_112.png"> corresponds to estimation on exponential window of effective length 10 samples.<p> 
    105 Statistics <img class="formulaInl" alt="$ V_0 , \nu_0 $" src="form_113.png"> are called alternative statistics, their role is to stabilize estimation. It is easy to show that for zero data, the statistics <img class="formulaInl" alt="$ V_t , \nu_t $" src="form_114.png"> converge to the alternative statistics.<h2><a class="anchor" name="str"> 
     104 Hence, <img class="formulaInl" alt="$ \phi=0.9 $" src="form_119.png"> corresponds to estimation on exponential window of effective length 10 samples.<p> 
     105Statistics <img class="formulaInl" alt="$ V_0 , \nu_0 $" src="form_120.png"> are called alternative statistics, their role is to stabilize estimation. It is easy to show that for zero data, the statistics <img class="formulaInl" alt="$ V_t , \nu_t $" src="form_121.png"> converge to the alternative statistics.<h2><a class="anchor" name="str"> 
    106106Structure estimation</a></h2> 
    107 For this model, structure estimation is a form of model selection procedure. Specifically, we compare hypotheses that the data were generated by the full model with hypotheses that some regressors in vector <img class="formulaInl" alt="$\psi$" src="form_13.png"> are redundant. The number of possible hypotheses is then the number of all possible combinations of all regressors.<p> 
    108 However, due to property known as nesting in exponential family, these hypotheses can be tested using only the posterior statistics. (This property does no hold for forgetting <img class="formulaInl" alt="$ \phi<1 $" src="form_115.png">). Hence, for low dimensional problems, this can be done by a tree search (method <a class="el" href="classbdm_1_1ARX.html#16b02ae03316751664c22d59d90c1e34" title="Brute force structure estimation.">bdm::ARX::structure_est()</a>). Or more sophisticated algorithm [ref Ludvik]<h2><a class="anchor" name="soft"> 
     107For this model, structure estimation is a form of model selection procedure. Specifically, we compare hypotheses that the data were generated by the full model with hypotheses that some regressors in vector <img class="formulaInl" alt="$\psi$" src="form_15.png"> are redundant. The number of possible hypotheses is then the number of all possible combinations of all regressors.<p> 
     108However, due to property known as nesting in exponential family, these hypotheses can be tested using only the posterior statistics. (This property does no hold for forgetting <img class="formulaInl" alt="$ \phi<1 $" src="form_122.png">). Hence, for low dimensional problems, this can be done by a tree search (method <a class="el" href="classbdm_1_1ARX.html#16b02ae03316751664c22d59d90c1e34" title="Brute force structure estimation.">bdm::ARX::structure_est()</a>). Or more sophisticated algorithm [ref Ludvik]<h2><a class="anchor" name="soft"> 
    109109Software Image</a></h2> 
    110110Estimation of the ARX model is implemented in class <a class="el" href="classbdm_1_1ARX.html" title="Linear Autoregressive model with Gaussian noise.">bdm::ARX</a>. <ul> 
    … …  
    123123</ul> 
    124124</div> 
    125 <hr size="1"><address style="text-align: right;"><small>Generated on Thu Apr 9 14:33:20 2009 for mixpp by&nbsp; 
     125<hr size="1"><address style="text-align: right;"><small>Generated on Thu Apr 23 21:06:43 2009 for mixpp by&nbsp; 
    126126<a href="http://www.doxygen.org/index.html"> 
    127127<img src="doxygen.png" alt="doxygen" align="middle" border="0"></a> 1.5.8 </small></address>