root/library/doc/tutorial/tut_arx.dox @ 680

/*!
\page tut_arx Theory of ARX model estimation
\addindex Theory of ARX estimation

The \c ARX (AutoregRessive with eXogeneous input) model is defined as follows:
\f[
y_t = \theta' \psi_t + \rho e_t 
\f]
where \f$y_t\f$ is the system output, \f$[\theta,\rho]\f$ is vector of unknown parameters, \f$\psi_t\f$ is an 
vector of data-dependent regressors, and noise \f$e_t\f$ is assumed to be Normal distributed \f$\mathcal{N}(0,1)\f$.

Special cases include:
\li estimation of unknown mean and variance of a Gaussian density from independent samples.

\section off Off-line estimation:
This particular model belongs to the exponential family, hence it has conjugate distribution (i.e. both prior and posterior) of the Gauss-inverse-Wishart form. See [ref]

Estimation of this family can be achieved by accumulation of sufficient statistics. The sufficient statistics  Gauss-inverse-Wishart density is composed of:
<dl>
<dt>Information matrix</dt> <dd>which is a sum of outer products \f[ 
V_t = \sum_{i=0}^{n} \left[\begin{array}{c}y_{t}\\ \psi_{t}\end{array}\right]
\begin{array}{c} [y_{t}',\,\psi_{t}']\\ \\\end{array}
\f]</dd>
<dt>"Degree of freedom"</dt>  <dd>which is an accumulator of number of data records \f[
\nu_t = \sum_{i=0}^{n} 1
\f]</dd>
</dl>

\section on On-line estimation
For online estimation with stationary parameters can be easily achieved by collecting the sufficient statistics described above recursively.

Extension to non-stationaly parameters, \f$ \theta_t , r_t \f$ can be achieved by operation called forgetting. This is an approximation of Bayesian filtering see [Kulhavy]. The resulting algorithm is defined by manipulation of sufficient statistics:
<dl>
<dt>Information matrix</dt> <dd>which is a sum of outer products \f[ 
V_t = \phi V_{t-1} +  \left[\begin{array}{c}y_{t}\\ \psi_{t}\end{array}\right]
\begin{array}{c} [y_{t}',\,\psi_{t}']\\ \\\end{array}
+(1-\phi) V_0
\f]
</dd>
<dt>"Degree of freedom"</dt>  <dd>which is an accumulator of number of data records \f[
\nu_t = \phi \nu_{t-1} + 1 + (1-\phi) \nu_0
\f]
</dd>
</dl>
where \f$ \phi \f$ is the forgetting factor, typically \f$ \phi \in [0,1]\f$ roughly corresponding to the effective length of the exponential window by relation:\f[
\mathrm{win_length} = \frac{1}{1-\phi}\f]
Hence, \f$ \phi=0.9 \f$ corresponds to estimation on exponential window of effective length 10 samples.

Statistics \f$ V_0 , \nu_0 \f$ are called alternative statistics, their role is to stabilize estimation. It is easy to show that for zero data, the statistics \f$ V_t , \nu_t \f$ converge to the alternative statistics. 

\section str Structure estimation
For this model, structure estimation is a form of model selection procedure.
Specifically, we compare hypotheses that the data were generated by the full model with hypotheses that some regressors in vector \f$\psi\f$ are redundant. The number of possible hypotheses is then the number of all possible combinations of all regressors.

However, due to property known as nesting in exponential family, these hypotheses can be tested using only the posterior statistics. (This property does no hold for forgetting \f$ \phi<1 \f$). Hence, for low dimensional problems, this can be done by a tree search (method bdm::ARX::structure_est()). Or more sophisticated algorithm [ref Ludvik]

\section soft Software Image
Estimation of the ARX model is implemented in class bdm::ARX.
\li models from exponential family share some properties, these are encoded in class bdm::BMEF which is the parent of ARX
\li one of the parameters of bdm::BMEF is the forgetting factor which is stored in attribute \c frg,
\li posterior density is stored inside the estimator in the form of bdm::egiw 
\li references to statistics of the internal \c egiw class, i.e. attributes \c V and \c nu  are established for convenience.

\section try How to try 
The best way to experiment with this object is to run matlab script \c  arx_test.m located in directory \c ./library/tutorial. See \ref arx_ui for detailed description.

\li In default setup, the parameters converge to the true values as expected.
\li Try changing the forgetting factor, field \c estimator.frg, to values <1. You should see increased lower and upper bounds on the estimates.
\li Try different set of parameters, filed \c system.theta, you should note that poles close to zero are harder to identify.


*/