| 1 | <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> |
|---|
| 2 | <html><head><meta http-equiv="Content-Type" content="text/html;charset=UTF-8"> |
|---|
| 3 | <title>mixpp: Unit testing of BDM</title> |
|---|
| 4 | <link href="tabs.css" rel="stylesheet" type="text/css"> |
|---|
| 5 | <link href="doxygen.css" rel="stylesheet" type="text/css"> |
|---|
| 6 | </head><body> |
|---|
| 7 | <!-- Generated by Doxygen 1.5.9 --> |
|---|
| 8 | <script type="text/javascript"> |
|---|
| 9 | <!-- |
|---|
| 10 | function changeDisplayState (e){ |
|---|
| 11 | var num=this.id.replace(/[^[0-9]/g,''); |
|---|
| 12 | var button=this.firstChild; |
|---|
| 13 | var sectionDiv=document.getElementById('dynsection'+num); |
|---|
| 14 | if (sectionDiv.style.display=='none'||sectionDiv.style.display==''){ |
|---|
| 15 | sectionDiv.style.display='block'; |
|---|
| 16 | button.src='open.gif'; |
|---|
| 17 | }else{ |
|---|
| 18 | sectionDiv.style.display='none'; |
|---|
| 19 | button.src='closed.gif'; |
|---|
| 20 | } |
|---|
| 21 | } |
|---|
| 22 | function initDynSections(){ |
|---|
| 23 | var divs=document.getElementsByTagName('div'); |
|---|
| 24 | var sectionCounter=1; |
|---|
| 25 | for(var i=0;i<divs.length-1;i++){ |
|---|
| 26 | if(divs[i].className=='dynheader'&&divs[i+1].className=='dynsection'){ |
|---|
| 27 | var header=divs[i]; |
|---|
| 28 | var section=divs[i+1]; |
|---|
| 29 | var button=header.firstChild; |
|---|
| 30 | if (button!='IMG'){ |
|---|
| 31 | divs[i].insertBefore(document.createTextNode(' '),divs[i].firstChild); |
|---|
| 32 | button=document.createElement('img'); |
|---|
| 33 | divs[i].insertBefore(button,divs[i].firstChild); |
|---|
| 34 | } |
|---|
| 35 | header.style.cursor='pointer'; |
|---|
| 36 | header.onclick=changeDisplayState; |
|---|
| 37 | header.id='dynheader'+sectionCounter; |
|---|
| 38 | button.src='closed.gif'; |
|---|
| 39 | section.id='dynsection'+sectionCounter; |
|---|
| 40 | section.style.display='none'; |
|---|
| 41 | section.style.marginLeft='14px'; |
|---|
| 42 | sectionCounter++; |
|---|
| 43 | } |
|---|
| 44 | } |
|---|
| 45 | } |
|---|
| 46 | window.onload = initDynSections; |
|---|
| 47 | --> |
|---|
| 48 | </script> |
|---|
| 49 | <div class="navigation" id="top"> |
|---|
| 50 | <div class="tabs"> |
|---|
| 51 | <ul> |
|---|
| 52 | <li><a href="main.html"><span>Main Page</span></a></li> |
|---|
| 53 | <li class="current"><a href="pages.html"><span>Related Pages</span></a></li> |
|---|
| 54 | <li><a href="annotated.html"><span>Classes</span></a></li> |
|---|
| 55 | <li><a href="files.html"><span>Files</span></a></li> |
|---|
| 56 | </ul> |
|---|
| 57 | </div> |
|---|
| 58 | <div class="navpath"><a class="el" href="dev_guide2.html">BDM Development - Contribution guide</a> |
|---|
| 59 | </div> |
|---|
| 60 | </div> |
|---|
| 61 | <div class="contents"> |
|---|
| 62 | <h1><a class="anchor" name="unit_testing">Unit testing of BDM </a></h1>Unit tests are simple, fast and easy to run tests of small pieces of code (the "units") designed to reassure developers that individual parts of the developped functionality behave as they're meant to. BDM being an object-oriented library, the independent units are classes, tested by calling their individual methods and comparing actual to expected results.<p> |
|---|
| 63 | Unit tests are especially useful for C++, where the simplest change can introduce (or uncover) a difficult-to-find bug. They also facilitate changing the internal structure of classes without modifying their external behavior - unit tests can be run before and after such a change to make sure the behavior had indeed stayed the same. More ambitiously, tests can be written before implementing new functionality, serving as its machine-checkable specification. Some authorities go so far as to demand that "not a single line of code" be written before having a failing unit test for it first, but BDM unit testing isn't that demanding - it does facilitate the style for developers who prefer it, but most BDM code is tested retroactively, with new tests suggested by various considerations (see below).<h2><a class="anchor" name="Setup"> |
|---|
| 64 | Setup</a></h2> |
|---|
| 65 | Unit tests of BDM use a 3rd-party unit testing framework, UnitTest++ (from <a href="http://unittest-cpp.sourceforge.net/">http://unittest-cpp.sourceforge.net/</a> ), which is designed specifically for C++, reasonably popular and maintained (for a C++ unit testing framework), works on Unix, Windows and Mac, allows simple test addition and proved to be readily extensible. Sources of UnitTest++ are included in directory library/tests/unittest-cpp, so that they can be built as part of the normal BDM build, without the need for explicit installation and setup, and are lightly modified for specific requirements of BDM.<p> |
|---|
| 66 | Existing unit tests are collected into a testsuite executable (so that they can all easily be run together), whose sources are in library/tests. Apart from the top-level testsuite.cpp, testsuite sources are generally named <classname>_test.cpp, where <classname> is the class tested by the file (or a base class of classes tested by the file, when these derived classes are similar enough to be tested in the same way). BDM doesn't test abstract classes using mock-ups (although it does test some template instantiations on test-only classes) - all its abstract classes already have concrete children and methods of the base class can be tested on them.<p> |
|---|
| 67 | When run, testsuite prints to standard output the names of performed tests and error messages when they fail, which means, among other things, that it should generally be run from the command line - even on Windows. Note that while UnitTest++ does have support for more elaborate GUIs, this support hasn't been extended for BDM-specific use cases. Testsuite can also be run under debugging tools - not only under a debugger, but also valgrind for checking memory management, coverage tools etc.<p> |
|---|
| 68 | Specific tests can be run individually, by passing their name (or names) as a command-line argument to testsuite. Note that a test which behaves differently depending on which tests run before it is a bug in testsuite (this can happen e.g. when using RVs, which are global - it's probably best to include the tested class name in their names, although not all existing tests do that yet). Tests of sampling and other functionality depending on random inputs can occasionally fail. Whether that's due to skewed inputs, BDM bugs or too tight error limits is not clear. Testsuite aims for every test passing with at least 95% probability, but the quantification of error limits isn't finished yet.<p> |
|---|
| 69 | Not all tests depending on UnitTest++ are in testsuite. Performance experiments have their own executables - currently just square_mat_stress, for testing BDM matrix wrappers. square_mat_stress is more configurable than testsuite: it doesn't generate its own random data, but reads them from a configuration file (named agenda.cfg by default) generated by another executable, square_mat_prep. This allows generation (and checking) of test inputs for various numerically demanding scenarios, which are encapsulated in square_mat_prep's generators - that is, classes implementing its generator interface. generators themselves are configurable using the same scripting framework as other BDM classes, and both square_mat_prep and square_mat_stress have command-line parameters - run them with '-?' to see the list. Analogical support can be created for other classes which would benefit from it. Focus of stress tests is rather different from unit tests as generally understood - that a class has stress tests doesn't mean it shouldn't be tested separately inside testsuite.<h2><a class="anchor" name="new_unit_tests"> |
|---|
| 70 | Adding new unit tests</a></h2> |
|---|
| 71 | Adding new unit tests is encouraged - it's a good way to get acquainted with the library, and there's always space for more tests.<p> |
|---|
| 72 | Unit tests are implemented by adding a block of code (it actually isn't just a function body, but it looks like one) starting with the TEST macro, whose argument is the test name:<p> |
|---|
| 73 | <div class="fragment"><pre class="fragment">TEST ( test_egiw ) { |
|---|
| 74 | epdf_harness::test_config ( <span class="stringliteral">"egiw.cfg"</span> ); |
|---|
| 75 | } |
|---|
| 76 | </pre></div><p> |
|---|
| 77 | "UnitTest++.h" should be included in every test file to have the macro defined.<p> |
|---|
| 78 | Tests are usually named test_<classname>, or test_<classname>_postfix when a class has more than one test:<p> |
|---|
| 79 | <div class="fragment"><pre class="fragment">TEST ( test_egiw_1_2 ) { |
|---|
| 80 | </pre></div><p> |
|---|
| 81 | This is useful when a class should be thoroughly tested from multiple aspects, e.g. for different dimensions. Exception-throwing execution paths are also generally run by extra tests, whose names end with _error. Note that each test in testsuite must have a unique name, which must be a legal C++ identifier.<p> |
|---|
| 82 | Testsuite has explicit support for testing classes derived from <a class="el" href="classbdm_1_1epdf.html" title="Probability density function with numerical statistics, e.g. posterior density.">bdm::epdf</a> and <a class="el" href="classbdm_1_1mpdf.html" title="Conditional probability density, e.g. modeling , where is random variable, rv, and...">bdm::mpdf</a>: bdm::epdf_harness and bdm::mpdf_harness. These classes run a list of tests on objects created from the specified configuration file (normally called <classname>.cfg) and check them against expected values also specified in that file, so that a test using harness is just a single-line call to its test_config method (as above).<p> |
|---|
| 83 | Unit tests can also be written explicitly. The test body is normal C++ code, implementing the checks of actual against expected values using various CHECK_* macros defined by UnitTest++:<p> |
|---|
| 84 | <div class="fragment"><pre class="fragment"> <span class="keywordtype">double</span> summ = 0.0; |
|---|
| 85 | <span class="keywordflow">for</span> ( <span class="keywordtype">int</span> k = 0; k < n; k++ ) { <span class="comment">// ALL b</span> |
|---|
| 86 | ... |
|---|
| 87 | } |
|---|
| 88 | |
|---|
| 89 | CHECK_CLOSE ( 1.0, summ, 0.1 ); |
|---|
| 90 | </pre></div><p> |
|---|
| 91 | Note that when checking vectors or matrices, you should also include "mat_checks.h", which defines their comparison. Unit tests shouldn't require any user interaction and probably shouldn't output anything other than error reports via the UnitTest++ facilities - among other things, UnitTest++ uses C functions for its output, and therefore a test printing via std::cout and std::cerr may report in rather confusing mixed order.<p> |
|---|
| 92 | Developers wishing to add new tests can use various heuristics to come up with the specific functionality to test - a selection is described below, roughly in the order of usefulness.<h3><a class="anchor" name="existing_bugs"> |
|---|
| 93 | Eliciting existing bugs</a></h3> |
|---|
| 94 | When the library has a bug, it may be useful to isolate the smallest possible code snippet which exhibits it - if not always, then at least as reliably as possible. Such a snippet is a good candidate for a unit test - it can be added either before the bug is fixed, or afterwards.<h3><a class="anchor" name="tricky_parts"> |
|---|
| 95 | Testing the code known to be tricky</a></h3> |
|---|
| 96 | Every body of code has its dark corners - copy&pasted legacy code, highly optimized algorithms, complicated interactions. These should be tested first.<h3><a class="anchor" name="unclear_behavior"> |
|---|
| 97 | Documenting and stabilizing unclear behavior</a></h3> |
|---|
| 98 | The library can behave in ways surprising its users without being necessarily "wrong". In such cases, it's most important that all maintainers agree on what the correct behavior should be (so that attempts to fix it don't run in circles). Unit tests constraining the contested functionality communicate very efficiently what their author thinks the behavior should (or shouldn't) be.<h3><a class="anchor" name="coverage"> |
|---|
| 99 | Increasing test coverage</a></h3> |
|---|
| 100 | Ideally, unit tests should test every method of every class (there are higher ideals, but they're even less realistic). Failing that, at least some methods of every class should be tested. Coverage tools (e.g. gcov) can show which parts of the library are not exercised by testsuite. </div> |
|---|
| 101 | <hr size="1"><address style="text-align: right;"><small>Generated on Wed Oct 7 17:34:45 2009 for mixpp by |
|---|
| 102 | <a href="http://www.doxygen.org/index.html"> |
|---|
| 103 | <img src="doxygen.png" alt="doxygen" align="middle" border="0"></a> 1.5.9 </small></address> |
|---|
| 104 | </body> |
|---|
| 105 | </html> |
|---|
