Index: library/doc/tutorial/userguide.dox
===================================================================
--- library/doc/tutorial/userguide.dox (revision 613)
+++ library/doc/tutorial/userguide.dox (revision 613)
@@ -0,0 +1,228 @@
+/*!
+\page user_guide Howto Use BDM - Introduction
+\addindex Howto Use BDM - Introduction
+
+BDM is a library of basic components for Bayesian decision making, hence its direct use is not possible. In order to use BDM the components must be pulled together in order to achieve desired functionality. We expect two kinds of users:
+
+ - <b> Basic users </b> who run prepared scripts with different parameterizations and analyze their results,
+ - <b> Advanced users </b> who are able to understand the logic of BDM and extend its functionality to new applications.
+
+The primary design aim of BDM was to ease development of complex algorithms, hence the target user is the advanced one. 
+However, running experiments is the first task to learn for both types of users.
+
+\section param Experiment is fully parameterized before execution
+
+Experiments in BDM can be performed using either standalone applications or function bindings in high-level environment. A typical example of the latter being mex file in Matlab environment.
+
+The main logic behind the experiment is that all necessary information about it are gathered in advance in a configuration file (for standalone applications) or in configuration structure (Matlab).
+This approach was designed especially for time consuming  experiments and Monte-Carlo studies for which it suits the most. 
+
+For smaller decision making tasks, interactive use of the experiment can be achieved by showing the full configuration structure (or its selected parts), running the experiment on demand and showing the results.
+
+Semi-interactive experiments can be designed by sequential run of different algorithms. This topic will be covered in advanced documentation.
+
+\section config Configuration of an experiment
+
+Configuration file (or config structure) is organized as a tree of information. High levels represent bigger structures, leafs of the structures are basic data elements such as strings, numbers or vectors.
+
+Specific treatment was developed for objects. Since BDM is designed as object oriented library, the configuration was designed to honor the rule of inheritance. That is, offspring of a class can be used in place of its predecessor. Hence, objects (instances of classes) are configured by a structure  with compulsory field \c class. This is a string variable corresponding to the name of the class to be used.
+
+Consider the following example:
+\code
+DS = {class="MemDS";
+   data = [1, 2, 3, 4, 5, 6, 7];
+}
+\endcode
+or written equivalently in Matlab as
+\code
+DS.class='MemDS';
+DS.Data =[1 2 3 4 5 6];
+\endcode
+
+The code above is the minimum necessary information to run a pre-made algorithm implemented as executable \c estimator or Matlab mex file \c estimator. The expected result for Matlab is:
+\code
+>> M=estimator(DS,{})
+
+M = 
+
+    ch0: [6x1 double]
+\endcode
+
+The structure \c M has one field called \c ch0 to which the data from \c DS.Data were copied. This was configured to be the default behavior which can be easily changed by adding more information to the configuration structure.
+
+First, we will have a look at all options of MemDS.
+
+\section memds How to understand configuration of classes
+
+As a first step, the estimator algorithm has created an object of class MemDS and called its method  bdm::MemDS::from_setting().
+This is a universal method called when creating an instance of class from configuration. Object that does not implement this method can not be created automatically from configuration.
+
+The documentation contains the full structure which can be loaded. e.g.:
+\code
+{ class = 'MemDS';
+	Data = (...);            // Data matrix or data vector
+	--- optional ---
+	drv = {class='RV'; ...} // Identification how rows of the matrix Data will be known to others
+	time = 0;               // Index of the first column to user_info,
+	rowid = [1,2,3...];     // ids of rows to be used
+}
+\endcode
+for MemDS. The compulsory fields are listed at the beginning; the optional fields are separated by string "--- optional ---".
+
+For the example given above, the missing fields were filled as follows:
+\code
+  drv  = {class="RV"; names="{ch0 }"; sizes=[1];};
+  time = 0;
+  rowid = [1];
+\endcode
+Meaning that the data will be read from the first column (time=0), all rows of data are to be read (rowid=[1]), and this row will be called "ch0". 
+
+\note <b>Mixtools reference</b> This object replaces global variables DATA and TIME. In BDM, data can be read and written to a range of \c datasources, objects derived from bdm::DS.
+
+\section rvs What is RV and how to use it
+
+RV stands for \c random \c variable which is a description of random variable or its realization. This object playes role of identifier of elements of vectors of data (in datasources), expected inputs to functions (in pdfs), or required results (operations conditioning). 
+
+\note <b>Mixtools reference </b> RV is generalization of "structures" \c str in Mixtools. It replaces channel numbers by string names, and adds extra field size for each record.
+
+Mathematical interpretation of RV is straightforward. Consider pdf \f$ f(a)\f$, then \f$ a \f$ is the part represented by RV. Explicit naming of random variables may seem unnecessary for many operations with pdf, e.g. for generation of a uniform sample from <0,1> it is not necessary to specify any random variable. For this reason, RV are often optional information to specify. However, the considered algorithm \c estimator is build in a way that requires RV to be given.
+
+The \c estimator use-case expects to join the data source with an array of estimators, each of which declaring its input vector of data. The connection will be made automatically using the mechanism of datalinks (bdm::datalink).
+Readers familiar with Simulink environment may look at the RV as being unique identifiers of inputs and outputs of simulation blocks. The inputs are connected automatically with the outputs with matching RV. This view is however, very incomplete, RV are much more powerful than this. 
+
+\section datasource Class inheritance and DataSources
+
+As mentioned above, the algorithm \c estimator is written to accept any datasource (i.e. any offspring of bdm::DS). For full list of offsprings, click Classes > Class Hierarchy.
+
+At the time of writing this tutorial, available datasources are
+bdm::DS
+ - bdm::EpdfDS
+ - bdm::MemDS
+   - bdm::FileDS
+     - bdm::CsvFileDS
+     - bdm::ITppFileDS
+ - bdm::MpdfDS
+ - bdm::stateDS
+
+The MemDS has already been introduced in the example in \ref memds.
+However, any of the classes listed above can be used to replace it in the example.
+This will be demonstrated on the \c EpdfDS class.
+
+Brief decription of the class states that EpdfDS "Simulate data from a static pdf (epdf)". The static pdf means unconditional pdf in the sense that the random variable is conditioned by numerical values only. In mathematical notation it could be both \f$ f(a) \f$ and \f$ f(x_t |d_1 \ldots d_t)\f$. The latter case is true only when all \f$ d \f$ denotes observed values.
+
+For example, we wish to simulate realizations of a Uniform density on interval <-1,1>. Uniform density is represented by class bdm::euni.
+From bdm::euni.from_setting() we can find that the code is:
+\code 
+U={class="euni"; high=1.0; low = -1.0;}
+\endcode
+for configuration file, and 
+\code
+U.class='euni';
+U.high = 1.0;
+U.low  = -1.0;
+U.rv.class = 'RV';
+U.rv.names = {'a'};
+\endcode
+for Matlab.
+  
+The datasource itself, can be then configured via
+\code
+DS = {class='EpdfDS'; epdf=@U;};
+\endcode
+in config file, or 
+\code
+DS.class = 'EpdfDS';
+DS.epdf  = U;
+\endcode
+in Matlab.
+
+Contrary to the previous example, we need to tell to algorithm \c estimator how many samples from the data source we need. This is configured by variable \c experiment.ndat. The configuration has to be finalized by:
+\code 
+experiment.ndat = 10;
+M=estimator(DS,{},experiment);
+\endcode
+
+The result is as expected in field \c M.a the name of which corresponds to name of \c U.rv .
+
+If the task was only to generate random realizations, this would indeed be a very clumsy way of doing it. However, the power of the proposed approach will be revelead in more demanding examples, one of which follows next.
+
+\section arx Simulating autoregressive model
+
+Consider the following autoregressive model: 
+\f[
+y_t \sim \mathcal{N}( a y_{t-3} + b u_{t-1}, r)
+\f]
+where \f$ a,b \f$ are known constants, and \f$ r \f$ is known variance.
+
+Direct application of \c EpdfDS is not possible, since the pdf above is conditioned on values of \f$ y_{t-3}\f$ and \f$ u_{t-1}\f$.
+We need to handle two issues:
+ -# extra unsimulated variable \f$ u \f$,
+ -# time delayes of the values.
+
+The first issue can be handled in two ways. First, \f$ u \f$ can be considered as input and as such it could be externally given to the datasource. This solution is used in algorithm use-case \c closedloop.
+However, for the \c estimator scenario we will apply the second option, that is we complement \f$ f(y_{t}|y_{t-3},u_{t-1})\f$ by extra pdf:\f[
+u_t \sim \mathcal{N}(0, r_u)
+\f]
+Thus, the joint density is now:\f[
+f(y_{t},u_{t}|y_{t-3},u_{t-1}) = f(y_{t}|y_{t-3},u_{t-1})f(u_{t})
+\f]
+and we have no need for input since the datasource have all necessary information inside. All that is required is to store them and copy their values to appropriate places.
+
+That is done in automatic way using dedicated class bdm::datalink_buffered. The only issue a user may need to take care about is the missing initial conditions for simulation.
+By default these are set to zeros. Using the default values, the full configuration of this system is:
+\code
+y = RV({'y'});
+u = RV({'u'});
+
+fy.class = 'mlnorm<ldmat>';
+fy.rv    = y;
+fy.rvc   = RV({'y','u'}, [1 1], [-3, -1]);
+fy.A     = [0.5, -0.9];
+fy.const = 0;
+fy.R     = 0.1;
+
+
+fu.class = 'enorm<ldmat>';
+fu.rv    = u;
+fu.mu    = 0;
+fu.R     = 0.2;
+
+DS.class = 'MpdfDS';
+DS.mpdf.class  = 'mprod';
+DS.mpdf.mpdfs  = {fy, epdf2mpdf(fu)};
+\endcode
+
+Explanation of this example will require few remarks:
+ - class of the \c fy object is 'mlnorm<ldmat>' which is Normal pdf with mean value given by linear function, and covariance matrix stored in LD decomposition, see bdm::mlnorm for details.
+ - naming convention 'mlnorm<ldmat>' relates to the concept of templates in C++. For those unfamiliar with this concept, it is basicaly a way how to share code for different flavours of the same object. Note that mlnorm exist in three versions: mlnorm<ldmat>, mlnorm<chmat>, mlnorm<fsqmat>. Those classes act identically the only difference is that the internal data are stored either in LD decomposition, choleski decomposition or full matrices, respectively. 
+ - the same concept is used for enorm, where enorm<chmat> and enorm<fsqmat> are also possible. In this particular use, these objects are equivalent. In specific situation, e.g. Kalman filter implemented on Choleski decomposition (bdm::KalmanCh), only enorm<chmat> is approprate.
+ - class 'mprod' represents the chain rule of probability. Attribute \c mpdfs of its configuration structure is a list of conditional densities. Conditional density \f$ f(a|b)\f$ is represented by class \c mpdf and its offsprings. Class \c RV is used to describe both variables before conditioning (field \c rv ) and after conditioning sign (field \c rvc).
+ - due to simplicity of implementation, mprod accept only conditional densities in the field \c mpdfs. Hence, the pdf \f$ f(u_t)\f$ must be converted to conditional density with empty conditioning, \f$ f(u_t| \{\})\f$. This is achieved by calling function epdf2mpdf which is only a trivial wrapper creating class bdm::mepdf.
+ 
+ 
+The code above can be immediatelly run, usin the same execution sequence of \c estimator as above. 
+
+\subsection ini Initializing simulation
+
+When zeros are not appropriate initial conditions, the correct conditions can be set using additional commands:
+\code
+DS.init_rv = RV({'y','y','y'}, [1,1,1], [-1,-2,-3]);
+DS.init_values = [0.1, 0.2, 0.3];
+\endcode
+
+The values of \c init_values will be copied to places in history identified by corresponding values of \c init_rv.
+Initial data is not checked for completeness, i.e. values of random variables missing from \c init_rv (in this case all occurences of \f$ u \f$) are still initialized to 0.
+
+\section conc What was demonstrated in this tutorial
+The purpose of this page was to introduce software image of basic elements of decision making as implemented in BDM.
+
+ - random values as identification mechanism (bdm::RV)
+ - unconditional pdfs (bdm::epdf),
+ - conditional pdfs (bdm::mpdf),
+ 
+And the use of these in simulation of data and function of datasources. In the next tutorial, Bayesian models (bdm::BM) and loggers (bdm::logger) will be introduced.
+
+
+
+
+*/
Index: library/doc/tutorial/devguide.dox
===================================================================
--- library/doc/tutorial/devguide.dox (revision 613)
+++ library/doc/tutorial/devguide.dox (revision 613)
@@ -0,0 +1,25 @@
+/*!
+\page dev_guide Howto Use BDM in C++ for your needs
+\addindex Howto Use BDM in C++ for your needs
+
+
+\section Intro Logic of BDM
+- \subpage intro
+- \subpage ui_page
+- \subpage mexfiles
+
+\section itpp Linear algebra library IT++
+
+- \subpage vector_and_matrix
+- \subpage itfile
+- \subpage timer
+
+\section bdm_direct Using BDM objects directly in C++
+
+See examples:
+
+- \subpage arx_ui
+- \subpage kalman
+
+
+*/
Index: library/doc/tutorial/memory_management.dox
===================================================================
--- library/doc/tutorial/memory_management.dox (revision 613)
+++ library/doc/tutorial/memory_management.dox (revision 613)
@@ -0,0 +1,140 @@
+/*!
+\page memory_management Memory Management in BDM
+
+C++ memory management is notoriously flexible, allowing a wide range
+of efficient and dangerous techniques. BDM uses conventions which
+allow high implementation efficiency (not absolutely maximal, but
+within a measurement error of the most efficient way) while
+substantially reducing the danger of memory errors. These conventions
+are described below.
+
+\section Constructors
+
+\li Each configurable class must have public default constructor (that
+is, a constructor which takes no parameters). This is required by the
+configuration framework. Other constructors may also be defined when
+convenient.
+
+\li Each constructor must initialize all fields of the constructed
+object, to prevent unpredictable behavior.
+
+One consequence of the points above is the case when a default
+constructor doesn't have the data with which to initialize a new
+object - in that case it can simply call the default constructors of
+all its object fields (and base classes), but must explicitly
+initialize all numeric fields and raw pointers to 0. Such an object
+isn't valid as constructed and must have some additional
+initialization methods (typically from_settings, for reading its
+configured state), but it can at least be destroyed.
+
+\section Exceptions
+
+BDM uses exceptions to signal runtime and some logic errors. The
+library aims to provide the minimal exception safety (that is,
+throwing an exception doesn't crash and doesn't leak any resources)
+for all thrown exceptions \b except memory errors - when a program
+using BDM exhausts memory, it should be terminated as soon as possible
+(and in most cases it has probably already terminated by
+itself). Specific exceptions may provide stronger guarantees, as
+documented for specific cases. All exceptions thrown out of the
+library are descendants of std::exception. Since they're organized
+into a class hierarchy, they should be caught by reference:
+
+\code
+try {
+	mpdf* mtmp = UI::build<mpdf> (_Sources, i);
+	Sources (i) = mtmp;
+} catch (UIException &exc) {
+\endcode
+
+This saves one call to copy constructor and prevents slicing.
+
+\section Pointers
+
+Pointers are used extensively (for efficiency), but usage of raw
+pointers should be minimized.
+
+Objects allocated by operator new should be assigned to a smart
+pointer instance immediately upon their construction, so that they can
+be automatically deleted after use. BDM implements its own
+reference-counted smart pointer template, bdm::shared_ptr, whose
+interface and semantics are close to the proposed standard
+std::tr1:shared_ptr (which is planned to replace bdm::shared_ptr once
+it becomes widely available). Note that objects allocated on the stack
+\b must \b not have their addresses passed to shared_ptr - that is a
+bug leading to intermittent runtime errors.
+
+Non-null heap pointers may also be kept in instances of object_ptr,
+which is convertible to (and from) shared_ptr. The SHAREDPTR macro (or
+SHAREDPTR2, for templated types) defines standartized names for these
+instances, simplifying library usage:
+
+\code
+/*
+ egamma_ptr is typedef for object_ptr<egamma>, whose default
+ constructor calls new egamma()
+*/
+egamma_ptr eG; 
+eG->set_parameters ( a, b );
+
+epdf_array Coms ( 2 ); // epdf_array is typedef for Array<shared_ptr<epdf> >
+Coms ( 0 ) = eG; // object_ptr<T> is derived from shared_ptr<T>
+
+/*
+ The egamma instance doesn't leak: if the shared_ptr instance which
+ wraps it isn't assigned to anything else, the pointer is deleted by
+ the destructor of either object_ptr, or Array, whichever runs last.
+*/
+\endcode
+
+Pointers kept in object fields should be wrapped in a shared_ptr
+instance, which will automatically keep them valid (at least) for the
+lifetime of the containing object. When that isn't possible, it should
+be documented why their containing class doesn't delete them and who
+does.
+
+Pointers passed as arguments into functions (and methods) are
+generally not expected to stay valid after the function returns - when
+that is required, the parameter's documentation should specify the
+required scope:
+
+\code
+/* the pointer must stay valid for the lifetime of the object */
+CurrentContext ( const char *name, int idx );
+\endcode
+
+A simpler alternative is just to pass a shared pointer:
+
+\code
+class mepdf : public mpdf
+{
+	shared_ptr<epdf> iepdf;
+public:
+	mepdf (shared_ptr<epdf> em) {
+		iepdf = em;
+\endcode
+
+In the case above, passing a (constant) reference to shared_ptr<epdf>
+might be more efficient, but no measurements have been performed.
+
+Functions returning raw pointers should document the scope of their
+validity:
+
+\code
+/*
+ Returns the stored pointer (which remains owned by this
+ instance).
+*/
+T *get();
+\endcode
+
+Functions generally shouldn't return raw pointers allocated by
+operator new - such pointers should be wrapped in an instance of
+shared_ptr, so that the pointer's unlimited life expectancy is encoded
+in the function signature:
+
+\code
+virtual shared_ptr<epdf> marginal (const RV &rv) const;
+\endcode
+
+*/
Index: library/doc/tutorial/install.dox
===================================================================
--- library/doc/tutorial/install.dox (revision 613)
+++ library/doc/tutorial/install.dox (revision 613)
@@ -0,0 +1,90 @@
+/*!
+\page install How to install BDM
+
+BDM is written as a multiplatform library, which was tested on Linux, Windows and Mac OSX.
+However, installation on each platform is specific.
+
+BDM comes in two basic versions:
+\li \b binary   with precompiled executables and matlab mex files that can be configured by external text files
+\li \b source \b code  that can be compiled into any required form. 
+Distribution of the binary form is not currently available. It is planned in latter stages of development.
+
+
+\section src Source code
+
+is available from svn repository:
+http://mys.utia.cas.cz:1800/svn/mixpp using guest/guest as login/password.
+
+For download use an appropriate SVN client http://subversion.tigris.org/links.html#clients
+
+Download the source files to any directory of your choise, this directoory will be refered to as \<bdm_dir\>.
+
+\subsection pre Prerequisities
+
+You need to have the following tools available on your system:
+\li working C++ compiler, e.g. gcc on Linux, or Visual Studio on windows
+\li CMake, a cross-platform build system, http://www.cmake.org
+\li ITPP library, http://itpp.sourceforge.net/ which is a part of all major linux distributions, for its installation on Windows, see Section Windows.
+ 
+\section lin Linux
+
+All prerequisities are available as binary packages for most distribution. E.g. for debian-based distributions, use
+\code
+apt-get install g++ cmake libitpp-dev 
+\endcode
+
+\note If you want to use mex files, you need to build IT++ from sources using the classi configure;make;make install. 
+The necessary flags for configure for mex compilation are in file linux/configure_itpp.
+
+Compile the BDM library and its applications using 
+\code
+$ cd <bdm_dir>/library/
+$ cmake .
+$ make
+\endcode
+
+
+\section win Windows
+
+The following approach was tested on Visual Studio 9, so we recommend to download and install it. 
+http://www.microsoft.com/express/vc/
+
+Make sure you have cmake installed.
+
+Expert users are advised to download and compile their ITPP version following a guide on : 
+http://itpp.sourceforge.net/current/installation.html
+
+For convenience of the beginners, DLLS of the ACML and lib+include files for ITPP and ACML are
+provided at: http://staff.utia.cas.cz/smidl/Public/mixpp_win32lib.zip
+If you want to use them copy the content of the zip file to \<bdm_dir\>/library/system/win32/ directory and
+make sure to set your PATH environment variable:
+\code
+ Path = %PATH%; <bdm_dir>\library\system\win32\dll
+\endcode
+Otherwise you will not be able to run any programs.
+
+
+Compilation of BDM:
+
+ * When everything is set-up, make a copy of library\system\win32\run_cmake.bat, edit it 
+  to reflect your environment (i.e. change paths to cmake and your C++ target environment) 
+  and run it.
+
+ * It should generate build files for your compiler in the main BDM 
+  directory
+
+\section mac Mac OS X
+
+All prerequisities for Mac OS X are avaliable except for ITPP, make sure you have them installed.
+
+However, its compilation is straightforward. 
+
+ 1. Download the latest stable release from http://itpp.sourceforge.net
+ 2. Compile it using script system/mac/itpp_compile.sh
+
+The default cmake configuration is looking for itpp in directory mac/itpp, make sure you have symlinks to this directory.
+
+If itpp compilation is successfull, you can run Cmake. Fill in the paths to your BDM directory, run Configure followed by Create.
+Configuration files for your favourite build environment should be created in the BDM directory.
+
+*/
Index: library/doc/tutorial/unit_testing.dox
===================================================================
--- library/doc/tutorial/unit_testing.dox (revision 613)
+++ library/doc/tutorial/unit_testing.dox (revision 613)
@@ -0,0 +1,183 @@
+/*!
+\page unit_testing Unit testing of BDM
+
+Unit tests are simple, fast and easy to run tests of small pieces of
+code (the "units") designed to reassure developers that individual
+parts of the developped functionality behave as they're meant to. BDM
+being an object-oriented library, the independent units are classes,
+tested by calling their individual methods and comparing actual to
+expected results.
+
+Unit tests are especially useful for C++, where the simplest change
+can introduce (or uncover) a difficult-to-find bug. They also
+facilitate changing the internal structure of classes without
+modifying their external behavior - unit tests can be run before and
+after such a change to make sure the behavior had indeed stayed the
+same. More ambitiously, tests can be written before implementing new
+functionality, serving as its machine-checkable specification. Some
+authorities go so far as to demand that "not a single line of code" be
+written before having a failing unit test for it first, but BDM unit
+testing isn't that demanding - it does facilitate the style for
+developers who prefer it, but most BDM code is tested retroactively,
+with new tests suggested by various considerations (see below).
+
+\section Setup
+
+Unit tests of BDM use a 3rd-party unit testing framework, UnitTest++
+(from http://unittest-cpp.sourceforge.net/ ), which is designed
+specifically for C++, reasonably popular and maintained (for a C++
+unit testing framework), works on Unix, Windows and Mac, allows simple
+test addition and proved to be readily extensible. Sources of
+UnitTest++ are included in directory library/tests/unittest-cpp, so
+that they can be built as part of the normal BDM build, without the
+need for explicit installation and setup, and are lightly modified for
+specific requirements of BDM.
+
+Existing unit tests are collected into a testsuite executable (so that
+they can all easily be run together), whose sources are in
+library/tests. Apart from the top-level testsuite.cpp, testsuite
+sources are generally named &lt;classname&gt;_test.cpp, where
+&lt;classname&gt; is the class tested by the file (or a base class of
+classes tested by the file, when these derived classes are similar
+enough to be tested in the same way). BDM doesn't test abstract
+classes using mock-ups (although it does test some template
+instantiations on test-only classes) - all its abstract classes
+already have concrete children and methods of the base class can be
+tested on them.
+
+When run, testsuite prints to standard output the names of performed
+tests and error messages when they fail, which means, among other
+things, that it should generally be run from the command line - even
+on Windows. Note that while UnitTest++ does have support for more
+elaborate GUIs, this support hasn't been extended for BDM-specific use
+cases. Testsuite can also be run under debugging tools - not only
+under a debugger, but also valgrind for checking memory management,
+coverage tools etc.
+
+Specific tests can be run individually, by passing their name (or
+names) as a command-line argument to testsuite. Note that a test which
+behaves differently depending on which tests run before it is a bug in
+testsuite (this can happen e.g. when using RVs, which are global -
+it's probably best to include the tested class name in their names,
+although not all existing tests do that yet). Tests of sampling and
+other functionality depending on random inputs can occasionally fail.
+Whether that's due to skewed inputs, BDM bugs or too tight error
+limits is not clear. Testsuite aims for every test passing with at
+least 95% probability, but the quantification of error limits isn't
+finished yet.
+
+Not all tests depending on UnitTest++ are in testsuite. Performance
+experiments have their own executables - currently just
+square_mat_stress, for testing BDM matrix wrappers. square_mat_stress
+is more configurable than testsuite: it doesn't generate its own
+random data, but reads them from a configuration file (named
+agenda.cfg by default) generated by another executable,
+square_mat_prep. This allows generation (and checking) of test inputs
+for various numerically demanding scenarios, which are encapsulated in
+square_mat_prep's generators - that is, classes implementing its
+generator interface. generators themselves are configurable using the
+same scripting framework as other BDM classes, and both
+square_mat_prep and square_mat_stress have command-line parameters -
+run them with '-?' to see the list. Analogical support can be created
+for other classes which would benefit from it. Focus of stress tests
+is rather different from unit tests as generally understood - that a
+class has stress tests doesn't mean it shouldn't be tested separately
+inside testsuite.
+
+\section new_unit_tests Adding new unit tests
+
+Adding new unit tests is encouraged - it's a good way to get
+acquainted with the library, and there's always space for more
+tests.
+
+Unit tests are implemented by adding a block of code (it actually
+isn't just a function body, but it looks like one) starting with the
+TEST macro, whose argument is the test name:
+
+\code
+TEST ( test_egiw ) {
+	epdf_harness::test_config ( "egiw.cfg" );
+}
+\endcode
+
+"UnitTest++.h" should be included in every test file to have the macro
+defined.
+
+Tests are usually named test_&lt;classname&gt;, or
+test_&lt;classname&gt;_postfix when a class has more than one test:
+
+\code
+TEST ( test_egiw_1_2 ) {
+\endcode
+
+This is useful when a class should be thoroughly tested from multiple
+aspects, e.g. for different dimensions. Exception-throwing execution
+paths are also generally run by extra tests, whose names end with
+_error. Note that each test in testsuite must have a unique name,
+which must be a legal C++ identifier.
+
+Testsuite has explicit support for testing classes derived from
+bdm::epdf and bdm::mpdf: bdm::epdf_harness and
+bdm::mpdf_harness. These classes run a list of tests on objects
+created from the specified configuration file (normally called
+&lt;classname&gt;.cfg) and check them against expected values also
+specified in that file, so that a test using harness is just a
+single-line call to its test_config method (as above).
+
+Unit tests can also be written explicitly. The test body is normal C++
+code, implementing the checks of actual against expected values using
+various CHECK_* macros defined by UnitTest++:
+
+\code
+	double summ = 0.0;
+	for ( int k = 0; k < n; k++ ) { // ALL b
+...
+	}
+
+	CHECK_CLOSE ( 1.0, summ, 0.1 );
+\endcode
+
+Note that when checking vectors or matrices, you should also include
+"mat_checks.h", which defines their comparison. Unit tests shouldn't
+require any user interaction and probably shouldn't output anything
+other than error reports via the UnitTest++ facilities - among other
+things, UnitTest++ uses C functions for its output, and therefore a
+test printing via std::cout and std::cerr may report in rather
+confusing mixed order.
+
+Developers wishing to add new tests can use various heuristics to come
+up with the specific functionality to test - a selection is described
+below, roughly in the order of usefulness.
+
+\subsection existing_bugs Eliciting existing bugs
+
+When the library has a bug, it may be useful to isolate the smallest
+possible code snippet which exhibits it - if not always, then at
+least as reliably as possible. Such a snippet is a good candidate for
+a unit test - it can be added either before the bug is fixed, or
+afterwards.
+
+\subsection tricky_parts Testing the code known to be tricky
+
+Every body of code has its dark corners - copy&pasted legacy code,
+highly optimized algorithms, complicated interactions. These should be
+tested first.
+
+\subsection unclear_behavior Documenting and stabilizing unclear behavior
+
+The library can behave in ways surprising its users without being
+necessarily "wrong". In such cases, it's most important that all
+maintainers agree on what the correct behavior should be (so that
+attempts to fix it don't run in circles). Unit tests constraining the
+contested functionality communicate very efficiently what their author
+thinks the behavior should (or shouldn't) be.
+
+\subsection coverage Increasing test coverage
+
+Ideally, unit tests should test every method of every class (there are
+higher ideals, but they're even less realistic). Failing that, at
+least some methods of every class should be tested. Coverage tools
+(e.g. gcov) can show which parts of the library are not exercised by
+testsuite.
+
+*/
Index: library/doc/tutorial/codingrules.dox
===================================================================
--- library/doc/tutorial/codingrules.dox (revision 613)
+++ library/doc/tutorial/codingrules.dox (revision 613)
@@ -0,0 +1,116 @@
+/*!
+\page codingrules Coding Rules (Mostly inherited from IT++)
+
+In the following sections we describe the naming conventions which are
+used for files, classes, structures, local variables, and global variables.
+
+
+\section cr_variables Default Naming Rules for Variables
+
+Generally, variables are named using lower-case letters and words are separated using
+under-score. But there are many exceptions, for instance abbreviations or classical 
+matematical notations. Therefore, coding rules for variables are quite free. Examples:
+
+<ul>
+<li> \c `FFT_size' </li>
+<li> \c `initial_RV' </li>
+<li> \c `my_variable' </li>
+</ul>
+
+\section cr_files Default Naming Rules for Files
+
+Files are named using lower-case letters and words are separated using
+under-score. If an abbreviation is inevitable within file name, it is written with 
+lower-case letters. 
+
+Source files are named using <tt>`.cpp'</tt> suffix, whereas header
+files end with <tt>`.h'</tt> extension. Examples:
+
+<ul>
+<li> <tt>`my_file.h'</tt> </li>
+<li> <tt>`user_info.cpp'</tt> </li>
+</ul>
+
+\section cr_file_templates Form of the source files
+
+For all the library classes, both header file `[filename].h' and source file 
+`[filename].cpp' should be implemented. And the following few rules should be 
+respected   
+
+<ul>
+
+<li> if possible, each `\#include ... ' dircetive should be located within the `.h'
+file, one obvious exception is the case of `\#include "[filename].h"' written in 
+`[filename].cpp' </li>
+
+<li> firstly, system headers should be included (i.e. those with brackets 
+`\#include \<system_header_name\>'), other headers (like `\#include "my_header_name.h"') 
+should follow (this rule leads to the faster search of an error on some 
+compilators) </li>
+
+<li> source code itself should be placed in the `.cpp' file, `.h' should 
+contains only class declarations and documentation (this rule has a few exceptions like
+ inline functions, templates and some extremely short function bodies)</li>
+
+</ul>
+
+Rules considering formatting of the source code itself are stored in the 
+file /system/astylerc, which is a configuration file for code formating 
+utility named ASTYLE. To apply them, download the proper version from 
+its web page http://astyle.sourceforge.net/     
+
+\section cr_functions Default Naming Rules for Functions
+
+Function names are named using lower-case letters and words are
+separated using under-score. Abbreviations, when used in function
+names, are also written with lower-case letters. This rule applies
+both to stand-alone functions as well as to member functions of
+classes. Example:
+
+<ul>
+<li> <tt>int my_function_name(int a, int b)</tt> </li>
+</ul>
+
+
+\section cr_specialfunctions Convention for sensitive functions
+
+For efficiency, some functions may return pointers to internal variables.
+Such functionality is indicated by underscore as the first letter in the
+the name.
+
+<ul>
+<li> <tt>mat* _internal_matrix()</tt> </li>
+</ul>
+
+
+\section cr_classes Default Naming Rules for Classes and Structures
+
+Each new word in a class or structure name should always start with a
+capital letter and the words should not be separated. Abbreviations are 
+written with capital letters. Examples:
+
+<ul>
+<li> \c `MyClassName' </li>
+<li> \c `MyStructName' </li>
+<li> \c `OFDM' </li>
+</ul>
+
+
+\section cr_classes_functionality Default Functionality of Classes
+
+All classes that are configured by input parameters should include:
+
+<ul>
+<li> default empty constructor </li>
+<li> one or more additional constructor(s) that takes input parameters
+and initializes the class instance </li>
+<li> setup function, preferably named \c `setup' or \c
+`set_parameters' </li>
+</ul>
+
+Explicit destructor functions are not required, unless they are
+needed. It shall not be possible to use any of the other member
+functions unless the class has been properly initiated with the input
+parameters.
+
+*/
Index: library/doc/tutorial/devguide2.dox
===================================================================
--- library/doc/tutorial/devguide2.dox (revision 613)
+++ library/doc/tutorial/devguide2.dox (revision 613)
@@ -0,0 +1,17 @@
+/*!
+\page dev_guide2 Howto Contribute to BDM - Advanced development
+\addindex Howto Contribute to BDM - Advanced development
+
+Get familiar with the library structure:
+\subpage library_structure
+
+You should read and follow the coding styles:
+\subpage codingrules
+
+Each contribution should come with elementary test of its functionality:
+\subpage unit_testing
+
+C++ is greatly flexible language which allows many tricks with memory management. For compatibility and ease of debugging, please follow guidelines given at:
+\subpage memory_management
+
+*/
Index: library/doc/tutorial/userguide2.dox
===================================================================
--- library/doc/tutorial/userguide2.dox (revision 613)
+++ library/doc/tutorial/userguide2.dox (revision 613)
@@ -0,0 +1,228 @@
+/*!
+\page user_guide Howto Use BDM - Introduction
+\addindex Howto Use BDM - Introduction
+
+BDM is a library of basic components for Bayesian decision making, hence its direct use is not possible. In order to use BDM the components must be pulled together in order to achieve desired functionality. We expect two kinds of users:
+
+ - <b> Basic users </b> who run prepared scripts with different parameterizations and analyze their results,
+ - <b> Advanced users </b> who are able to understand the logic of BDM and extend its functionality to new applications.
+
+The primary design aim of BDM was to ease development of complex algorithms, hence the target user is the advanced one. 
+However, running experiments is the first task to learn for both types of users.
+
+\section param Experiment is fully parameterized before execution
+
+Experiments in BDM can be performed using either standalone applications or function bindings in high-level environment. A typical example of the latter being mex file in Matlab environment.
+
+The main logic behind the experiment is that all necessary information about it are gathered in advance in a configuration file (for standalone applications) or in configuration structure (Matlab).
+This approach was designed especially for time consuming  experiments and Monte-Carlo studies for which it suits the most. 
+
+For smaller decision making tasks, interactive use of the experiment can be achieved by showing the full configuration structure (or its selected parts), running the experiment on demand and showing the results.
+
+Semi-interactive experiments can be designed by sequential run of different algorithms. This topic will be covered in advanced documentation.
+
+\section config Configuration of an experiment
+
+Configuration file (or config structure) is organized as a tree of information. High levels represent bigger structures, leafs of the structures are basic data elements such as strings, numbers or vectors.
+
+Specific treatment was developed for objects. Since BDM is designed as object oriented library, the configuration was designed to honor the rule of inheritance. That is, offspring of a class can be used in place of its predecessor. Hence, objects (instances of classes) are configured by a structure  with compulsory field \c class. This is a string variable corresponding to the name of the class to be used.
+
+Consider the following example:
+\code
+DS = {class="MemDS";
+   data = [1, 2, 3, 4, 5, 6, 7];
+}
+\endcode
+or written equivalently in Matlab as
+\code
+DS.class='MemDS';
+DS.Data =[1 2 3 4 5 6];
+\endcode
+
+The code above is the minimum necessary information to run a pre-made algorithm implemented as executable \c estimator or Matlab mex file \c estimator. The expected result for Matlab is:
+\code
+>> M=estimator(DS,{})
+
+M = 
+
+    ch0: [6x1 double]
+\endcode
+
+The structure \c M has one field called \c ch0 to which the data from \c DS.Data were copied. This was configured to be the default behavior which can be easily changed by adding more information to the configuration structure.
+
+First, we will have a look at all options of MemDS.
+
+\section memds How to understand configuration of classes
+
+As a first step, the estimator algorithm has created an object of class MemDS and called its method  bdm::MemDS::from_setting().
+This is a universal method called when creating an instance of class from configuration. Object that does not implement this method can not be created automatically from configuration.
+
+The documentation contains the full structure which can be loaded. e.g.:
+\code
+{ class = 'MemDS';
+	Data = (...);            // Data matrix or data vector
+	--- optional ---
+	drv = {class='RV'; ...} // Identification how rows of the matrix Data will be known to others
+	time = 0;               // Index of the first column to user_info,
+	rowid = [1,2,3...];     // ids of rows to be used
+}
+\endcode
+for MemDS. The compulsory fields are listed at the beginning; the optional fields are separated by string "--- optional ---".
+
+For the example given above, the missing fields were filled as follows:
+\code
+  drv  = {class="RV"; names="{ch0 }"; sizes=[1];};
+  time = 0;
+  rowid = [1];
+\endcode
+Meaning that the data will be read from the first column (time=0), all rows of data are to be read (rowid=[1]), and this row will be called "ch0". 
+
+\note <b>Mixtools reference</b> This object replaces global variables DATA and TIME. In BDM, data can be read and written to a range of \c datasources, objects derived from bdm::DS.
+
+\section rvs What is RV and how to use it
+
+RV stands for \c random \c variable which is a description of random variable or its realization. This object playes role of identifier of elements of vectors of data (in datasources), expected inputs to functions (in pdfs), or required results (operations conditioning). 
+
+\note <b>Mixtools reference </b> RV is generalization of "structures" \c str in Mixtools. It replaces channel numbers by string names, and adds extra field size for each record.
+
+Mathematical interpretation of RV is straightforward. Consider pdf \f$ f(a)\f$, then \f$ a \f$ is the part represented by RV. Explicit naming of random variables may seem unnecessary for many operations with pdf, e.g. for generation of a uniform sample from <0,1> it is not necessary to specify any random variable. For this reason, RV are often optional information to specify. However, the considered algorithm \c estimator is build in a way that requires RV to be given.
+
+The \c estimator use-case expects to join the data source with an array of estimators, each of which declaring its input vector of data. The connection will be made automatically using the mechanism of datalinks (bdm::datalink).
+Readers familiar with Simulink environment may look at the RV as being unique identifiers of inputs and outputs of simulation blocks. The inputs are connected automatically with the outputs with matching RV. This view is however, very incomplete, RV are much more powerful than this. 
+
+\section datasource Class inheritance and DataSources
+
+As mentioned above, the algorithm \c estimator is written to accept any datasource (i.e. any offspring of bdm::DS). For full list of offsprings, click Classes > Class Hierarchy.
+
+At the time of writing this tutorial, available datasources are
+bdm::DS
+ - bdm::EpdfDS
+ - bdm::MemDS
+   - bdm::FileDS
+     - bdm::CsvFileDS
+     - bdm::ITppFileDS
+ - bdm::MpdfDS
+ - bdm::stateDS
+
+The MemDS has already been introduced in the example in \ref memds.
+However, any of the classes listed above can be used to replace it in the example.
+This will be demonstrated on the \c EpdfDS class.
+
+Brief decription of the class states that EpdfDS "Simulate data from a static pdf (epdf)". The static pdf means unconditional pdf in the sense that the random variable is conditioned by numerical values only. In mathematical notation it could be both \f$ f(a) \f$ and \f$ f(x_t |d_1 \ldots d_t)\f$. The latter case is true only when all \f$ d \f$ denotes observed values.
+
+For example, we wish to simulate realizations of a Uniform density on interval <-1,1>. Uniform density is represented by class bdm::euni.
+From bdm::euni.from_setting() we can find that the code is:
+\code 
+U={class="euni"; high=1.0; low = -1.0;}
+\endcode
+for configuration file, and 
+\code
+U.class='euni';
+U.high = 1.0;
+U.low  = -1.0;
+U.rv.class = 'RV';
+U.rv.names = {'a'};
+\endcode
+for Matlab.
+  
+The datasource itself, can be then configured via
+\code
+DS = {class='EpdfDS'; epdf=@U;};
+\endcode
+in config file, or 
+\code
+DS.class = 'EpdfDS';
+DS.epdf  = U;
+\endcode
+in Matlab.
+
+Contrary to the previous example, we need to tell to algorithm \c estimator how many samples from the data source we need. This is configured by variable \c experiment.ndat. The configuration has to be finalized by:
+\code 
+experiment.ndat = 10;
+M=estimator(DS,{},experiment);
+\endcode
+
+The result is as expected in field \c M.a the name of which corresponds to name of \c U.rv .
+
+If the task was only to generate random realizations, this would indeed be a very clumsy way of doing it. However, the power of the proposed approach will be revelead in more demanding examples, one of which follows next.
+
+\section arx Simulating autoregressive model
+
+Consider the following autoregressive model: 
+\f[
+y_t \sim \mathcal{N}( a y_{t-3} + b u_{t-1}, r)
+\f]
+where \f$ a,b \f$ are known constants, and \f$ r \f$ is known variance.
+
+Direct application of \c EpdfDS is not possible, since the pdf above is conditioned on values of \f$ y_{t-3}\f$ and \f$ u_{t-1}\f$.
+We need to handle two issues:
+ -# extra unsimulated variable \f$ u \f$,
+ -# time delayes of the values.
+
+The first issue can be handled in two ways. First, \f$ u \f$ can be considered as input and as such it could be externally given to the datasource. This solution is used in algorithm use-case \c closedloop.
+However, for the \c estimator scenario we will apply the second option, that is we complement \f$ f(y_{t}|y_{t-3},u_{t-1})\f$ by extra pdf:\f[
+u_t \sim \mathcal{N}(0, r_u)
+\f]
+Thus, the joint density is now:\f[
+f(y_{t},u_{t}|y_{t-3},u_{t-1}) = f(y_{t}|y_{t-3},u_{t-1})f(u_{t})
+\f]
+and we have no need for input since the datasource have all necessary information inside. All that is required is to store them and copy their values to appropriate places.
+
+That is done in automatic way using dedicated class bdm::datalink_buffered. The only issue a user may need to take care about is the missing initial conditions for simulation.
+By default these are set to zeros. Using the default values, the full configuration of this system is:
+\code
+y = RV({'y'});
+u = RV({'u'});
+
+fy.class = 'mlnorm<ldmat>';
+fy.rv    = y;
+fy.rvc   = RV({'y','u'}, [1 1], [-3, -1]);
+fy.A     = [0.5, -0.9];
+fy.const = 0;
+fy.R     = 0.1;
+
+
+fu.class = 'enorm<ldmat>';
+fu.rv    = u;
+fu.mu    = 0;
+fu.R     = 0.2;
+
+DS.class = 'MpdfDS';
+DS.mpdf.class  = 'mprod';
+DS.mpdf.mpdfs  = {fy, epdf2mpdf(fu)};
+\endcode
+
+Explanation of this example will require few remarks:
+ - class of the \c fy object is 'mlnorm<ldmat>' which is Normal pdf with mean value given by linear function, and covariance matrix stored in LD decomposition, see bdm::mlnorm for details.
+ - naming convention 'mlnorm<ldmat>' relates to the concept of templates in C++. For those unfamiliar with this concept, it is basicaly a way how to share code for different flavours of the same object. Note that mlnorm exist in three versions: mlnorm<ldmat>, mlnorm<chmat>, mlnorm<fsqmat>. Those classes act identically the only difference is that the internal data are stored either in LD decomposition, choleski decomposition or full matrices, respectively. 
+ - the same concept is used for enorm, where enorm<chmat> and enorm<fsqmat> are also possible. In this particular use, these objects are equivalent. In specific situation, e.g. Kalman filter implemented on Choleski decomposition (bdm::KalmanCh), only enorm<chmat> is approprate.
+ - class 'mprod' represents the chain rule of probability. Attribute \c mpdfs of its configuration structure is a list of conditional densities. Conditional density \f$ f(a|b)\f$ is represented by class \c mpdf and its offsprings. Class \c RV is used to describe both variables before conditioning (field \c rv ) and after conditioning sign (field \c rvc).
+ - due to simplicity of implementation, mprod accept only conditional densities in the field \c mpdfs. Hence, the pdf \f$ f(u_t)\f$ must be converted to conditional density with empty conditioning, \f$ f(u_t| \{\})\f$. This is achieved by calling function epdf2mpdf which is only a trivial wrapper creating class bdm::mepdf.
+ 
+ 
+The code above can be immediatelly run, usin the same execution sequence of \c estimator as above. 
+
+\subsection ini Initializing simulation
+
+When zeros are not appropriate initial conditions, the correct conditions can be set using additional commands:
+\code
+DS.init_rv = RV({'y','y','y'}, [1,1,1], [-1,-2,-3]);
+DS.init_values = [0.1, 0.2, 0.3];
+\endcode
+
+The values of \c init_values will be copied to places in history identified by corresponding values of \c init_rv.
+Initial data is not checked for completeness, i.e. values of random variables missing from \c init_rv (in this case all occurences of \f$ u \f$) are still initialized to 0.
+
+\section conc What was demonstrated in this tutorial
+The purpose of this page was to introduce software image of basic elements of decision making as implemented in BDM.
+
+ - random values as identification mechanism (bdm::RV)
+ - unconditional pdfs (bdm::epdf),
+ - conditional pdfs (bdm::mpdf),
+ 
+And the use of these in simulation of data and function of datasources. In the next tutorial, Bayesian models (bdm::BM) and loggers (bdm::logger) will be introduced.
+
+
+
+
+*/
Index: library/doc/tutorial/library_structure.dox
===================================================================
--- library/doc/tutorial/library_structure.dox (revision 613)
+++ library/doc/tutorial/library_structure.dox (revision 613)
@@ -0,0 +1,32 @@
+/*!
+\page library_structure Library structure
+
+\todo it is necessary to fill in the missing descriptions of the directories below 
+
+Here you find infromation about the library structure. List of its 
+subdirectories with a basic description follows:
+<ul>
+<li> source:/bdm -- library source codes </li>
+<li> source:/bdm/base -- base classes of the library </li>
+<li> source:/bdm/design -- ... </li>
+<li> source:/bdm/estim -- ... </li>
+<li> source:/bdm/math -- ... </li>
+<li> source:/bdm/mex -- ... </li>
+<li> source:/bdm/stat -- ... </li>
+<li> source:/doc -- library documantation and with some of its sources </li>
+<li> source:/doc/dot -- ... </li>
+<li> source:/doc/html -- html documentation generated automatically by Doxygen </li>
+<li> source:/doc/images -- ... </li>
+<li> source:/doc/issues -- ... </li>
+<li> source:/doc/latex -- latex documentation generated automatically by Doxygen </li>
+<li> source:/doc/local -- documentation pages which are not related to any specific source file </li>
+<li> source:/doc/tutorial -- documentation pages related to the library tutorial</li>
+<li> source:/doc/xml -- ... </li>
+<li> source:/matlab -- matlab input/output utilities </li>
+<li> source:/system -- a system specific parts necessary to run CMake </li>
+<li> source:/tests -- unit tests </li>
+<li> source:/utia_legacy -- ... </li>
+</ul>
+
+
+*/
Index: library/doc/tutorial/manual.dox
===================================================================
--- library/doc/tutorial/manual.dox (revision 471)
+++  (revision )
@@ -1,26 +1,0 @@
-/*!
-\page manual User Manual
-\addindex User Manual
-
-This page contains some tutorial examples that will help you getting
-started using BDM. 
-
-Basic linear algebra in BDM (using IT++):
-- \subpage vector_and_matrix
-- \subpage itfile
-- \subpage timer
-
-Basic concepts and philosophy of BDM:
-- \subpage intro
-- \subpage ui_page
-- \subpage mexfiles
-
-Library of predefined tasks:
-
-Examples for Various mathematical models (BDM):
-- \subpage arx_ui
-- \subpage kalman
-
-Examples of control (BDM):
-- To be done...
-*/
