Show
Ignore:
Timestamp:
10/12/09 13:49:39 (15 years ago)
Author:
mido
Message:

\doc directory cleaned a bit

Files:
1 modified

Legend:

Unmodified
Added
Removed

  • library/doc/html/tut_arx.html

    r641 r651  
    1 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> 
    2 <html xmlns="http://www.w3.org/1999/xhtml"> 
    3 <head> 
    4 <meta http-equiv="Content-Type" content="text/xhtml;charset=UTF-8"/> 
     1<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> 
     2<html><head><meta http-equiv="Content-Type" content="text/html;charset=UTF-8"> 
    53<title>mixpp: Theory of ARX model estimation</title> 
    6 <link href="tabs.css" rel="stylesheet" type="text/css"/> 
    7 <link href="doxygen.css" rel="stylesheet" type="text/css"/> 
    8 </head> 
    9 <body> 
    10 <!-- Generated by Doxygen 1.6.1 --> 
     4<link href="tabs.css" rel="stylesheet" type="text/css"> 
     5<link href="doxygen.css" rel="stylesheet" type="text/css"> 
     6</head><body> 
     7<!-- Generated by Doxygen 1.5.9 --> 
    118<script type="text/javascript"> 
    129<!-- 
    … …  
    6360</div> 
    6461<div class="contents"> 
    65  
    66  
    67 <h1><a class="anchor" id="tut_arx">Theory of ARX model estimation </a></h1><p></p> 
    68 <p>The <code>ARX</code> (AutoregRessive with eXogeneous input) model is defined as follows: </p> 
    69 <p class="formulaDsp"> 
    70 <img class="formulaDsp" alt="\[ y_t = \theta' \psi_t + \rho e_t \]" src="form_126.png"/> 
    71 </p> 
    72 <p> where <img class="formulaInl" alt="$y_t$" src="form_9.png"/> is the system output, <img class="formulaInl" alt="$[\theta,\rho]$" src="form_127.png"/> is vector of unknown parameters, <img class="formulaInl" alt="$\psi_t$" src="form_128.png"/> is an vector of data-dependent regressors, and noise <img class="formulaInl" alt="$e_t$" src="form_32.png"/> is assumed to be Normal distributed <img class="formulaInl" alt="$\mathcal{N}(0,1)$" src="form_129.png"/>.</p> 
    73 <p>Special cases include: </p> 
    74 <ul> 
     62<h1><a class="anchor" name="tut_arx">Theory of ARX model estimation </a></h1><p> 
     63The <code>ARX</code> (AutoregRessive with eXogeneous input) model is defined as follows: <p class="formulaDsp"> 
     64<img class="formulaDsp" alt="\[ y_t = \theta' \psi_t + \rho e_t \]" src="form_126.png"> 
     65<p> 
     66 where <img class="formulaInl" alt="$y_t$" src="form_9.png"> is the system output, <img class="formulaInl" alt="$[\theta,\rho]$" src="form_127.png"> is vector of unknown parameters, <img class="formulaInl" alt="$\psi_t$" src="form_128.png"> is an vector of data-dependent regressors, and noise <img class="formulaInl" alt="$e_t$" src="form_32.png"> is assumed to be Normal distributed <img class="formulaInl" alt="$\mathcal{N}(0,1)$" src="form_129.png">.<p> 
     67Special cases include: <ul> 
    7568<li>estimation of unknown mean and variance of a Gaussian density from independent samples.</li> 
    7669</ul> 
    77 <h2><a class="anchor" id="off"> 
     70<h2><a class="anchor" name="off"> 
    7871Off-line estimation:</a></h2> 
    79 <p>This particular model belongs to the exponential family, hence it has conjugate distribution (i.e. both prior and posterior) of the Gauss-inverse-Wishart form. See [ref]</p> 
    80 <p>Estimation of this family can be achieved by accumulation of sufficient statistics. The sufficient statistics Gauss-inverse-Wishart density is composed of: </p> 
    81 <dl> 
     72This particular model belongs to the exponential family, hence it has conjugate distribution (i.e. both prior and posterior) of the Gauss-inverse-Wishart form. See [ref]<p> 
     73Estimation of this family can be achieved by accumulation of sufficient statistics. The sufficient statistics Gauss-inverse-Wishart density is composed of: <dl> 
    8274<dt>Information matrix </dt> 
    8375<dd>which is a sum of outer products <p class="formulaDsp"> 
    84 <img class="formulaDsp" alt="\[ V_t = \sum_{i=0}^{n} \left[\begin{array}{c}y_{t}\\ \psi_{t}\end{array}\right] \begin{array}{c} [y_{t}',\,\psi_{t}']\\ \\\end{array} \]" src="form_130.png"/> 
    85 </p> 
     76<img class="formulaDsp" alt="\[ V_t = \sum_{i=0}^{n} \left[\begin{array}{c}y_{t}\\ \psi_{t}\end{array}\right] \begin{array}{c} [y_{t}',\,\psi_{t}']\\ \\\end{array} \]" src="form_130.png"> 
     77<p> 
    8678 </dd> 
    8779<dt>"Degree of freedom" </dt> 
    8880<dd>which is an accumulator of number of data records <p class="formulaDsp"> 
    89 <img class="formulaDsp" alt="\[ \nu_t = \sum_{i=0}^{n} 1 \]" src="form_131.png"/> 
    90 </p> 
     81<img class="formulaDsp" alt="\[ \nu_t = \sum_{i=0}^{n} 1 \]" src="form_131.png"> 
     82<p> 
    9183 </dd> 
    9284</dl> 
    93 <h2><a class="anchor" id="on"> 
     85<h2><a class="anchor" name="on"> 
    9486On-line estimation</a></h2> 
    95 <p>For online estimation with stationary parameters can be easily achieved by collecting the sufficient statistics described above recursively.</p> 
    96 <p>Extension to non-stationaly parameters, <img class="formulaInl" alt="$ \theta_t , r_t $" src="form_132.png"/> can be achieved by operation called forgetting. This is an approximation of Bayesian filtering see [Kulhavy]. The resulting algorithm is defined by manipulation of sufficient statistics: </p> 
    97 <dl> 
     87For online estimation with stationary parameters can be easily achieved by collecting the sufficient statistics described above recursively.<p> 
     88Extension to non-stationaly parameters, <img class="formulaInl" alt="$ \theta_t , r_t $" src="form_132.png"> can be achieved by operation called forgetting. This is an approximation of Bayesian filtering see [Kulhavy]. The resulting algorithm is defined by manipulation of sufficient statistics: <dl> 
    9889<dt>Information matrix </dt> 
    9990<dd>which is a sum of outer products <p class="formulaDsp"> 
    100 <img class="formulaDsp" alt="\[ V_t = \phi V_{t-1} + \left[\begin{array}{c}y_{t}\\ \psi_{t}\end{array}\right] \begin{array}{c} [y_{t}',\,\psi_{t}']\\ \\\end{array} +(1-\phi) V_0 \]" src="form_133.png"/> 
    101 </p> 
     91<img class="formulaDsp" alt="\[ V_t = \phi V_{t-1} + \left[\begin{array}{c}y_{t}\\ \psi_{t}\end{array}\right] \begin{array}{c} [y_{t}',\,\psi_{t}']\\ \\\end{array} +(1-\phi) V_0 \]" src="form_133.png"> 
     92<p> 
    10293  </dd> 
    10394<dt>"Degree of freedom" </dt> 
    10495<dd>which is an accumulator of number of data records <p class="formulaDsp"> 
    105 <img class="formulaDsp" alt="\[ \nu_t = \phi \nu_{t-1} + 1 + (1-\phi) \nu_0 \]" src="form_134.png"/> 
    106 </p> 
     96<img class="formulaDsp" alt="\[ \nu_t = \phi \nu_{t-1} + 1 + (1-\phi) \nu_0 \]" src="form_134.png"> 
     97<p> 
    10798  </dd> 
    10899</dl> 
    109 <p>where <img class="formulaInl" alt="$ \phi $" src="form_135.png"/> is the forgetting factor, typically <img class="formulaInl" alt="$ \phi \in [0,1]$" src="form_136.png"/> roughly corresponding to the effective length of the exponential window by relation:</p> 
    110 <p class="formulaDsp"> 
    111 <img class="formulaDsp" alt="\[ \mathrm{win_length} = \frac{1}{1-\phi}\]" src="form_137.png"/> 
    112 </p> 
    113 <p> Hence, <img class="formulaInl" alt="$ \phi=0.9 $" src="form_138.png"/> corresponds to estimation on exponential window of effective length 10 samples.</p> 
    114 <p>Statistics <img class="formulaInl" alt="$ V_0 , \nu_0 $" src="form_139.png"/> are called alternative statistics, their role is to stabilize estimation. It is easy to show that for zero data, the statistics <img class="formulaInl" alt="$ V_t , \nu_t $" src="form_140.png"/> converge to the alternative statistics.</p> 
    115 <h2><a class="anchor" id="str"> 
     100where <img class="formulaInl" alt="$ \phi $" src="form_135.png"> is the forgetting factor, typically <img class="formulaInl" alt="$ \phi \in [0,1]$" src="form_136.png"> roughly corresponding to the effective length of the exponential window by relation:<p class="formulaDsp"> 
     101<img class="formulaDsp" alt="\[ \mathrm{win_length} = \frac{1}{1-\phi}\]" src="form_137.png"> 
     102<p> 
     103 Hence, <img class="formulaInl" alt="$ \phi=0.9 $" src="form_138.png"> corresponds to estimation on exponential window of effective length 10 samples.<p> 
     104Statistics <img class="formulaInl" alt="$ V_0 , \nu_0 $" src="form_139.png"> are called alternative statistics, their role is to stabilize estimation. It is easy to show that for zero data, the statistics <img class="formulaInl" alt="$ V_t , \nu_t $" src="form_140.png"> converge to the alternative statistics.<h2><a class="anchor" name="str"> 
    116105Structure estimation</a></h2> 
    117 <p>For this model, structure estimation is a form of model selection procedure. Specifically, we compare hypotheses that the data were generated by the full model with hypotheses that some regressors in vector <img class="formulaInl" alt="$\psi$" src="form_41.png"/> are redundant. The number of possible hypotheses is then the number of all possible combinations of all regressors.</p> 
    118 <p>However, due to property known as nesting in exponential family, these hypotheses can be tested using only the posterior statistics. (This property does no hold for forgetting <img class="formulaInl" alt="$ \phi<1 $" src="form_141.png"/>). Hence, for low dimensional problems, this can be done by a tree search (method <a class="el" href="classbdm_1_1ARX.html#a16b02ae03316751664c22d59d90c1e34" title="Brute force structure estimation.">bdm::ARX::structure_est()</a>). Or more sophisticated algorithm [ref Ludvik]</p> 
    119 <h2><a class="anchor" id="soft"> 
     106For this model, structure estimation is a form of model selection procedure. Specifically, we compare hypotheses that the data were generated by the full model with hypotheses that some regressors in vector <img class="formulaInl" alt="$\psi$" src="form_41.png"> are redundant. The number of possible hypotheses is then the number of all possible combinations of all regressors.<p> 
     107However, due to property known as nesting in exponential family, these hypotheses can be tested using only the posterior statistics. (This property does no hold for forgetting <img class="formulaInl" alt="$ \phi<1 $" src="form_141.png">). Hence, for low dimensional problems, this can be done by a tree search (method <a class="el" href="classbdm_1_1ARX.html#16b02ae03316751664c22d59d90c1e34" title="Brute force structure estimation.">bdm::ARX::structure_est()</a>). Or more sophisticated algorithm [ref Ludvik]<h2><a class="anchor" name="soft"> 
    120108Software Image</a></h2> 
    121 <p>Estimation of the ARX model is implemented in class <a class="el" href="classbdm_1_1ARX.html" title="Linear Autoregressive model with Gaussian noise.">bdm::ARX</a>. </p> 
    122 <ul> 
     109Estimation of the ARX model is implemented in class <a class="el" href="classbdm_1_1ARX.html" title="Linear Autoregressive model with Gaussian noise.">bdm::ARX</a>. <ul> 
    123110<li>models from exponential family share some properties, these are encoded in class <a class="el" href="classbdm_1_1BMEF.html" title="Estimator for Exponential family.">bdm::BMEF</a> which is the parent of ARX </li> 
    124111<li>one of the parameters of <a class="el" href="classbdm_1_1BMEF.html" title="Estimator for Exponential family.">bdm::BMEF</a> is the forgetting factor which is stored in attribute <code>frg</code>, </li> 
    … …  
    126113<li>references to statistics of the internal <code>egiw</code> class, i.e. attributes <code>V</code> and <code>nu</code> are established for convenience.</li> 
    127114</ul> 
    128 <h2><a class="anchor" id="try"> 
     115<h2><a class="anchor" name="try"> 
    129116How to try</a></h2> 
    130 <p>The best way to experiment with this object is to run matlab script <code>arx_test.m</code> located in directory <code></code>./library/tutorial. See <a class="el" href="arx_ui.html">Running experiment <code>estimator</code> with ARX data fields  this page is out of date, as the user info concept has been changed</a> for detailed description.</p> 
     117The best way to experiment with this object is to run matlab script <code>arx_test.m</code> located in directory <code></code>./library/tutorial. See <a class="el" href="arx_ui.html">Running experiment <code>estimator</code> with ARX data fields</a> for detailed description.<p> 
    131118<ul> 
    132119<li>In default setup, the parameters converge to the true values as expected. </li> 
    … …  
    135122</ul> 
    136123</div> 
    137 <hr size="1"/><address style="text-align: right;"><small>Generated on Sun Sep 27 00:49:05 2009 for mixpp by&nbsp; 
     124<hr size="1"><address style="text-align: right;"><small>Generated on Wed Oct 7 17:34:45 2009 for mixpp by&nbsp; 
    138125<a href="http://www.doxygen.org/index.html"> 
    139 <img class="footer" src="doxygen.png" alt="doxygen"/></a> 1.6.1 </small></address> 
     126<img src="doxygen.png" alt="doxygen" align="middle" border="0"></a> 1.5.9 </small></address> 
    140127</body> 
    141128</html>