root/doc/html/tut_arx.html @ 323

Revision 323, 8.6 kB (checked in by smidl, 16 years ago)

doc

RevLine 
[275]1<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
2<html><head><meta http-equiv="Content-Type" content="text/html;charset=UTF-8">
3<title>mixpp: Theory of ARX model estimation</title>
[290]4<link href="tabs.css" rel="stylesheet" type="text/css">
[275]5<link href="doxygen.css" rel="stylesheet" type="text/css">
6</head><body>
[290]7<!-- Generated by Doxygen 1.5.8 -->
[275]8<script type="text/javascript">
9<!--
10function changeDisplayState (e){
11  var num=this.id.replace(/[^[0-9]/g,'');
12  var button=this.firstChild;
13  var sectionDiv=document.getElementById('dynsection'+num);
14  if (sectionDiv.style.display=='none'||sectionDiv.style.display==''){
15    sectionDiv.style.display='block';
16    button.src='open.gif';
17  }else{
18    sectionDiv.style.display='none';
19    button.src='closed.gif';
20  }
21}
22function initDynSections(){
23  var divs=document.getElementsByTagName('div');
24  var sectionCounter=1;
25  for(var i=0;i<divs.length-1;i++){
26    if(divs[i].className=='dynheader'&&divs[i+1].className=='dynsection'){
27      var header=divs[i];
28      var section=divs[i+1];
29      var button=header.firstChild;
30      if (button!='IMG'){
31        divs[i].insertBefore(document.createTextNode(' '),divs[i].firstChild);
32        button=document.createElement('img');
33        divs[i].insertBefore(button,divs[i].firstChild);
34      }
35      header.style.cursor='pointer';
36      header.onclick=changeDisplayState;
37      header.id='dynheader'+sectionCounter;
38      button.src='closed.gif';
39      section.id='dynsection'+sectionCounter;
40      section.style.display='none';
41      section.style.marginLeft='14px';
42      sectionCounter++;
43    }
44  }
45}
46window.onload = initDynSections;
47-->
48</script>
49<div class="navigation" id="top">
50  <div class="tabs">
51    <ul>
52      <li><a href="main.html"><span>Main&nbsp;Page</span></a></li>
[290]53      <li class="current"><a href="pages.html"><span>Related&nbsp;Pages</span></a></li>
[275]54      <li><a href="modules.html"><span>Modules</span></a></li>
[290]55      <li><a href="annotated.html"><span>Classes</span></a></li>
[275]56      <li><a href="files.html"><span>Files</span></a></li>
57    </ul>
58  </div>
59  <div class="navpath"><a class="el" href="tutorial.html">Tutorial in Bayesian estimation</a>
60  </div>
61</div>
62<div class="contents">
63<h1><a class="anchor" name="tut_arx">Theory of ARX model estimation </a></h1><p>
64The <code>ARX</code> (AutoregRessive with eXogeneous input) model is defined as follows: <p class="formulaDsp">
[323]65<img class="formulaDsp" alt="\[ y_t = \theta' \psi_t + \rho e_t \]" src="form_107.png">
[275]66<p>
[323]67 where <img class="formulaInl" alt="$y_t$" src="form_42.png"> is the system output, <img class="formulaInl" alt="$[\theta,\rho]$" src="form_108.png"> is vector of unknown parameters, <img class="formulaInl" alt="$\psi_t$" src="form_109.png"> is an vector of data-dependent regressors, and noise <img class="formulaInl" alt="$e_t$" src="form_6.png"> is assumed to be Normal distributed <img class="formulaInl" alt="$\mathcal{N}(0,1)$" src="form_110.png">.<p>
[275]68Special cases include: <ul>
69<li>estimation of unknown mean and variance of a Gaussian density from independent samples.</li>
70</ul>
71<h2><a class="anchor" name="off">
72Off-line estimation:</a></h2>
73This particular model belongs to the exponential family, hence it has conjugate distribution (i.e. both prior and posterior) of the Gauss-inverse-Wishart form. See [ref]<p>
74Estimation of this family can be achieved by accumulation of sufficient statistics. The sufficient statistics Gauss-inverse-Wishart density is composed of: <dl>
75<dt>Information matrix </dt>
76<dd>which is a sum of outer products <p class="formulaDsp">
[323]77<img class="formulaDsp" alt="\[ V_t = \sum_{i=0}^{n} \left[\begin{array}{c}y_{t}\\ \psi_{t}\end{array}\right] \begin{array}{c} [y_{t}',\,\psi_{t}']\\ \\\end{array} \]" src="form_111.png">
[275]78<p>
79 </dd>
80<dt>"Degree of freedom" </dt>
81<dd>which is an accumulator of number of data records <p class="formulaDsp">
[323]82<img class="formulaDsp" alt="\[ \nu_t = \sum_{i=0}^{n} 1 \]" src="form_112.png">
[275]83<p>
84 </dd>
85</dl>
86<h2><a class="anchor" name="on">
87On-line estimation</a></h2>
88For online estimation with stationary parameters can be easily achieved by collecting the sufficient statistics described above recursively.<p>
[323]89Extension to non-stationaly parameters, <img class="formulaInl" alt="$ \theta_t , r_t $" src="form_113.png"> can be achieved by operation called forgetting. This is an approximation of Bayesian filtering see [Kulhavy]. The resulting algorithm is defined by manipulation of sufficient statistics: <dl>
[275]90<dt>Information matrix </dt>
91<dd>which is a sum of outer products <p class="formulaDsp">
[323]92<img class="formulaDsp" alt="\[ V_t = \phi V_{t-1} + \left[\begin{array}{c}y_{t}\\ \psi_{t}\end{array}\right] \begin{array}{c} [y_{t}',\,\psi_{t}']\\ \\\end{array} +(1-\phi) V_0 \]" src="form_114.png">
[275]93<p>
94 </dd>
95<dt>"Degree of freedom" </dt>
96<dd>which is an accumulator of number of data records <p class="formulaDsp">
[323]97<img class="formulaDsp" alt="\[ \nu_t = \phi \nu_{t-1} + 1 + (1-\phi) \nu_0 \]" src="form_115.png">
[275]98<p>
99 </dd>
100</dl>
[323]101where <img class="formulaInl" alt="$ \phi $" src="form_116.png"> is the forgetting factor, typically <img class="formulaInl" alt="$ \phi \in [0,1]$" src="form_117.png"> roughly corresponding to the effective length of the exponential window by relation:<p class="formulaDsp">
102<img class="formulaDsp" alt="\[ \mathrm{win_length} = \frac{1}{1-\phi}\]" src="form_118.png">
[275]103<p>
[323]104 Hence, <img class="formulaInl" alt="$ \phi=0.9 $" src="form_119.png"> corresponds to estimation on exponential window of effective length 10 samples.<p>
105Statistics <img class="formulaInl" alt="$ V_0 , \nu_0 $" src="form_120.png"> are called alternative statistics, their role is to stabilize estimation. It is easy to show that for zero data, the statistics <img class="formulaInl" alt="$ V_t , \nu_t $" src="form_121.png"> converge to the alternative statistics.<h2><a class="anchor" name="str">
[275]106Structure estimation</a></h2>
[323]107For this model, structure estimation is a form of model selection procedure. Specifically, we compare hypotheses that the data were generated by the full model with hypotheses that some regressors in vector <img class="formulaInl" alt="$\psi$" src="form_15.png"> are redundant. The number of possible hypotheses is then the number of all possible combinations of all regressors.<p>
108However, due to property known as nesting in exponential family, these hypotheses can be tested using only the posterior statistics. (This property does no hold for forgetting <img class="formulaInl" alt="$ \phi<1 $" src="form_122.png">). Hence, for low dimensional problems, this can be done by a tree search (method <a class="el" href="classbdm_1_1ARX.html#16b02ae03316751664c22d59d90c1e34" title="Brute force structure estimation.">bdm::ARX::structure_est()</a>). Or more sophisticated algorithm [ref Ludvik]<h2><a class="anchor" name="soft">
[275]109Software Image</a></h2>
110Estimation of the ARX model is implemented in class <a class="el" href="classbdm_1_1ARX.html" title="Linear Autoregressive model with Gaussian noise.">bdm::ARX</a>. <ul>
111<li>models from exponential family share some properties, these are encoded in class <a class="el" href="classbdm_1_1BMEF.html" title="Estimator for Exponential family.">bdm::BMEF</a> which is the parent of ARX </li>
112<li>one of the parameters of <a class="el" href="classbdm_1_1BMEF.html" title="Estimator for Exponential family.">bdm::BMEF</a> is the forgetting factor which is stored in attribute <code>frg</code>, </li>
113<li>posterior density is stored inside the estimator in the form of <a class="el" href="classbdm_1_1egiw.html" title="Gauss-inverse-Wishart density stored in LD form.">bdm::egiw</a> </li>
114<li>references to statistics of the internal <code>egiw</code> class, i.e. attributes <code>V</code> and <code>nu</code> are established for convenience.</li>
115</ul>
116<h2><a class="anchor" name="try">
117How to try</a></h2>
118The best way to experiment with this object is to run matlab script <code>arx_test.m</code> located in directory <code></code>./library/tutorial. See <a class="el" href="arx_ui.html">Running experiment <code>estimator</code> with ARX data fields</a> for detailed description.<p>
119<ul>
120<li>In default setup, the parameters converge to the true values as expected. </li>
121<li>Try changing the forgetting factor, field <code>estimator.frg</code>, to values &lt;1. You should see increased lower and upper bounds on the estimates. </li>
122<li>Try different set of parameters, filed <code>system.theta</code>, you should note that poles close to zero are harder to identify. </li>
123</ul>
124</div>
[323]125<hr size="1"><address style="text-align: right;"><small>Generated on Thu Apr 23 21:06:43 2009 for mixpp by&nbsp;
[275]126<a href="http://www.doxygen.org/index.html">
[290]127<img src="doxygen.png" alt="doxygen" align="middle" border="0"></a> 1.5.8 </small></address>
[275]128</body>
129</html>
Note: See TracBrowser for help on using the browser.