Show
Ignore:
Timestamp:
08/30/09 22:13:15 (15 years ago)
Author:
smidl
Message:

doc

Files:
1 modified

Legend:

Unmodified
Added
Removed

  • library/doc/html/tut_arx.html

    r590 r591  
    1 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> 
    2 <html><head><meta http-equiv="Content-Type" content="text/html;charset=UTF-8"> 
     1<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> 
     2<html xmlns="http://www.w3.org/1999/xhtml"> 
     3<head> 
     4<meta http-equiv="Content-Type" content="text/xhtml;charset=UTF-8"/> 
    35<title>mixpp: Theory of ARX model estimation</title> 
    4 <link href="tabs.css" rel="stylesheet" type="text/css"> 
    5 <link href="doxygen.css" rel="stylesheet" type="text/css"> 
    6 </head><body> 
    7 <!-- Generated by Doxygen 1.5.8 --> 
     6<link href="tabs.css" rel="stylesheet" type="text/css"/> 
     7<link href="doxygen.css" rel="stylesheet" type="text/css"/> 
     8</head> 
     9<body> 
     10<!-- Generated by Doxygen 1.6.1 --> 
    811<script type="text/javascript"> 
    912<!-- 
    … …  
    6063</div> 
    6164<div class="contents"> 
    62 <h1><a class="anchor" name="tut_arx">Theory of ARX model estimation </a></h1><p> 
    63 The <code>ARX</code> (AutoregRessive with eXogeneous input) model is defined as follows: <p class="formulaDsp"> 
    64 <img class="formulaDsp" alt="\[ y_t = \theta' \psi_t + \rho e_t \]" src="form_115.png"> 
    65 <p> 
    66  where <img class="formulaInl" alt="$y_t$" src="form_3.png"> is the system output, <img class="formulaInl" alt="$[\theta,\rho]$" src="form_116.png"> is vector of unknown parameters, <img class="formulaInl" alt="$\psi_t$" src="form_117.png"> is an vector of data-dependent regressors, and noise <img class="formulaInl" alt="$e_t$" src="form_24.png"> is assumed to be Normal distributed <img class="formulaInl" alt="$\mathcal{N}(0,1)$" src="form_118.png">.<p> 
    67 Special cases include: <ul> 
     65 
     66 
     67<h1><a class="anchor" id="tut_arx">Theory of ARX model estimation </a></h1><p></p> 
     68<p>The <code>ARX</code> (AutoregRessive with eXogeneous input) model is defined as follows: </p> 
     69<p class="formulaDsp"> 
     70<img class="formulaDsp" alt="\[ y_t = \theta' \psi_t + \rho e_t \]" src="form_115.png"/> 
     71</p> 
     72<p> where <img class="formulaInl" alt="$y_t$" src="form_3.png"/> is the system output, <img class="formulaInl" alt="$[\theta,\rho]$" src="form_116.png"/> is vector of unknown parameters, <img class="formulaInl" alt="$\psi_t$" src="form_117.png"/> is an vector of data-dependent regressors, and noise <img class="formulaInl" alt="$e_t$" src="form_24.png"/> is assumed to be Normal distributed <img class="formulaInl" alt="$\mathcal{N}(0,1)$" src="form_118.png"/>.</p> 
     73<p>Special cases include: </p> 
     74<ul> 
    6875<li>estimation of unknown mean and variance of a Gaussian density from independent samples.</li> 
    6976</ul> 
    70 <h2><a class="anchor" name="off"> 
     77<h2><a class="anchor" id="off"> 
    7178Off-line estimation:</a></h2> 
    72 This particular model belongs to the exponential family, hence it has conjugate distribution (i.e. both prior and posterior) of the Gauss-inverse-Wishart form. See [ref]<p> 
    73 Estimation of this family can be achieved by accumulation of sufficient statistics. The sufficient statistics Gauss-inverse-Wishart density is composed of: <dl> 
     79<p>This particular model belongs to the exponential family, hence it has conjugate distribution (i.e. both prior and posterior) of the Gauss-inverse-Wishart form. See [ref]</p> 
     80<p>Estimation of this family can be achieved by accumulation of sufficient statistics. The sufficient statistics Gauss-inverse-Wishart density is composed of: </p> 
     81<dl> 
    7482<dt>Information matrix </dt> 
    7583<dd>which is a sum of outer products <p class="formulaDsp"> 
    76 <img class="formulaDsp" alt="\[ V_t = \sum_{i=0}^{n} \left[\begin{array}{c}y_{t}\\ \psi_{t}\end{array}\right] \begin{array}{c} [y_{t}',\,\psi_{t}']\\ \\\end{array} \]" src="form_119.png"> 
    77 <p> 
     84<img class="formulaDsp" alt="\[ V_t = \sum_{i=0}^{n} \left[\begin{array}{c}y_{t}\\ \psi_{t}\end{array}\right] \begin{array}{c} [y_{t}',\,\psi_{t}']\\ \\\end{array} \]" src="form_119.png"/> 
     85</p> 
    7886 </dd> 
    7987<dt>"Degree of freedom" </dt> 
    8088<dd>which is an accumulator of number of data records <p class="formulaDsp"> 
    81 <img class="formulaDsp" alt="\[ \nu_t = \sum_{i=0}^{n} 1 \]" src="form_120.png"> 
    82 <p> 
     89<img class="formulaDsp" alt="\[ \nu_t = \sum_{i=0}^{n} 1 \]" src="form_120.png"/> 
     90</p> 
    8391 </dd> 
    8492</dl> 
    85 <h2><a class="anchor" name="on"> 
     93<h2><a class="anchor" id="on"> 
    8694On-line estimation</a></h2> 
    87 For online estimation with stationary parameters can be easily achieved by collecting the sufficient statistics described above recursively.<p> 
    88 Extension to non-stationaly parameters, <img class="formulaInl" alt="$ \theta_t , r_t $" src="form_121.png"> can be achieved by operation called forgetting. This is an approximation of Bayesian filtering see [Kulhavy]. The resulting algorithm is defined by manipulation of sufficient statistics: <dl> 
     95<p>For online estimation with stationary parameters can be easily achieved by collecting the sufficient statistics described above recursively.</p> 
     96<p>Extension to non-stationaly parameters, <img class="formulaInl" alt="$ \theta_t , r_t $" src="form_121.png"/> can be achieved by operation called forgetting. This is an approximation of Bayesian filtering see [Kulhavy]. The resulting algorithm is defined by manipulation of sufficient statistics: </p> 
     97<dl> 
    8998<dt>Information matrix </dt> 
    9099<dd>which is a sum of outer products <p class="formulaDsp"> 
    91 <img class="formulaDsp" alt="\[ V_t = \phi V_{t-1} + \left[\begin{array}{c}y_{t}\\ \psi_{t}\end{array}\right] \begin{array}{c} [y_{t}',\,\psi_{t}']\\ \\\end{array} +(1-\phi) V_0 \]" src="form_122.png"> 
    92 <p> 
     100<img class="formulaDsp" alt="\[ V_t = \phi V_{t-1} + \left[\begin{array}{c}y_{t}\\ \psi_{t}\end{array}\right] \begin{array}{c} [y_{t}',\,\psi_{t}']\\ \\\end{array} +(1-\phi) V_0 \]" src="form_122.png"/> 
     101</p> 
    93102  </dd> 
    94103<dt>"Degree of freedom" </dt> 
    95104<dd>which is an accumulator of number of data records <p class="formulaDsp"> 
    96 <img class="formulaDsp" alt="\[ \nu_t = \phi \nu_{t-1} + 1 + (1-\phi) \nu_0 \]" src="form_123.png"> 
    97 <p> 
     105<img class="formulaDsp" alt="\[ \nu_t = \phi \nu_{t-1} + 1 + (1-\phi) \nu_0 \]" src="form_123.png"/> 
     106</p> 
    98107  </dd> 
    99108</dl> 
    100 where <img class="formulaInl" alt="$ \phi $" src="form_124.png"> is the forgetting factor, typically <img class="formulaInl" alt="$ \phi \in [0,1]$" src="form_125.png"> roughly corresponding to the effective length of the exponential window by relation:<p class="formulaDsp"> 
    101 <img class="formulaDsp" alt="\[ \mathrm{win_length} = \frac{1}{1-\phi}\]" src="form_126.png"> 
    102 <p> 
    103  Hence, <img class="formulaInl" alt="$ \phi=0.9 $" src="form_127.png"> corresponds to estimation on exponential window of effective length 10 samples.<p> 
    104 Statistics <img class="formulaInl" alt="$ V_0 , \nu_0 $" src="form_128.png"> are called alternative statistics, their role is to stabilize estimation. It is easy to show that for zero data, the statistics <img class="formulaInl" alt="$ V_t , \nu_t $" src="form_129.png"> converge to the alternative statistics.<h2><a class="anchor" name="str"> 
     109<p>where <img class="formulaInl" alt="$ \phi $" src="form_124.png"/> is the forgetting factor, typically <img class="formulaInl" alt="$ \phi \in [0,1]$" src="form_125.png"/> roughly corresponding to the effective length of the exponential window by relation:</p> 
     110<p class="formulaDsp"> 
     111<img class="formulaDsp" alt="\[ \mathrm{win_length} = \frac{1}{1-\phi}\]" src="form_126.png"/> 
     112</p> 
     113<p> Hence, <img class="formulaInl" alt="$ \phi=0.9 $" src="form_127.png"/> corresponds to estimation on exponential window of effective length 10 samples.</p> 
     114<p>Statistics <img class="formulaInl" alt="$ V_0 , \nu_0 $" src="form_128.png"/> are called alternative statistics, their role is to stabilize estimation. It is easy to show that for zero data, the statistics <img class="formulaInl" alt="$ V_t , \nu_t $" src="form_129.png"/> converge to the alternative statistics.</p> 
     115<h2><a class="anchor" id="str"> 
    105116Structure estimation</a></h2> 
    106 For this model, structure estimation is a form of model selection procedure. Specifically, we compare hypotheses that the data were generated by the full model with hypotheses that some regressors in vector <img class="formulaInl" alt="$\psi$" src="form_33.png"> are redundant. The number of possible hypotheses is then the number of all possible combinations of all regressors.<p> 
    107 However, due to property known as nesting in exponential family, these hypotheses can be tested using only the posterior statistics. (This property does no hold for forgetting <img class="formulaInl" alt="$ \phi<1 $" src="form_130.png">). Hence, for low dimensional problems, this can be done by a tree search (method <a class="el" href="classbdm_1_1ARX.html#16b02ae03316751664c22d59d90c1e34" title="Brute force structure estimation.">bdm::ARX::structure_est()</a>). Or more sophisticated algorithm [ref Ludvik]<h2><a class="anchor" name="soft"> 
     117<p>For this model, structure estimation is a form of model selection procedure. Specifically, we compare hypotheses that the data were generated by the full model with hypotheses that some regressors in vector <img class="formulaInl" alt="$\psi$" src="form_33.png"/> are redundant. The number of possible hypotheses is then the number of all possible combinations of all regressors.</p> 
     118<p>However, due to property known as nesting in exponential family, these hypotheses can be tested using only the posterior statistics. (This property does no hold for forgetting <img class="formulaInl" alt="$ \phi<1 $" src="form_130.png"/>). Hence, for low dimensional problems, this can be done by a tree search (method <a class="el" href="classbdm_1_1ARX.html#a16b02ae03316751664c22d59d90c1e34" title="Brute force structure estimation.">bdm::ARX::structure_est()</a>). Or more sophisticated algorithm [ref Ludvik]</p> 
     119<h2><a class="anchor" id="soft"> 
    108120Software Image</a></h2> 
    109 Estimation of the ARX model is implemented in class <a class="el" href="classbdm_1_1ARX.html" title="Linear Autoregressive model with Gaussian noise.">bdm::ARX</a>. <ul> 
     121<p>Estimation of the ARX model is implemented in class <a class="el" href="classbdm_1_1ARX.html" title="Linear Autoregressive model with Gaussian noise.">bdm::ARX</a>. </p> 
     122<ul> 
    110123<li>models from exponential family share some properties, these are encoded in class <a class="el" href="classbdm_1_1BMEF.html" title="Estimator for Exponential family.">bdm::BMEF</a> which is the parent of ARX </li> 
    111124<li>one of the parameters of <a class="el" href="classbdm_1_1BMEF.html" title="Estimator for Exponential family.">bdm::BMEF</a> is the forgetting factor which is stored in attribute <code>frg</code>, </li> 
    … …  
    113126<li>references to statistics of the internal <code>egiw</code> class, i.e. attributes <code>V</code> and <code>nu</code> are established for convenience.</li> 
    114127</ul> 
    115 <h2><a class="anchor" name="try"> 
     128<h2><a class="anchor" id="try"> 
    116129How to try</a></h2> 
    117 The best way to experiment with this object is to run matlab script <code>arx_test.m</code> located in directory <code></code>./library/tutorial. See <a class="el" href="arx_ui.html">Running experiment <code>estimator</code> with ARX data fields</a> for detailed description.<p> 
     130<p>The best way to experiment with this object is to run matlab script <code>arx_test.m</code> located in directory <code></code>./library/tutorial. See <a class="el" href="arx_ui.html">Running experiment <code>estimator</code> with ARX data fields</a> for detailed description.</p> 
    118131<ul> 
    119132<li>In default setup, the parameters converge to the true values as expected. </li> 
    … …  
    122135</ul> 
    123136</div> 
    124 <hr size="1"><address style="text-align: right;"><small>Generated on Sat Aug 29 20:49:42 2009 for mixpp by&nbsp; 
     137<hr size="1"/><address style="text-align: right;"><small>Generated on Sun Aug 30 22:10:50 2009 for mixpp by&nbsp; 
    125138<a href="http://www.doxygen.org/index.html"> 
    126 <img src="doxygen.png" alt="doxygen" align="middle" border="0"></a> 1.5.8 </small></address> 
     139<img class="footer" src="doxygen.png" alt="doxygen"/></a> 1.6.1 </small></address> 
    127140</body> 
    128141</html>