# Unit Testing

If this project handbook would have been for a professional project (with professional I mean, a project that people make money with), I would have written

Unit tests must be supplied by the developer with the classes/source-code he checks into the repository!.

Since this is the handbook for a voluntary work (which is not less professional than any other project), I replace the above sentence with

Each developer in this project is strongly encouraged to develop unit tests for the code he or she develops and make them available to the project team!.

## Why unit testing?

Before I can give an answer to this question, I should explain what unit testing is about. I do not cover all relevant aspects here nor do I start a discussion of the various aspects of unit testing. If you want to read more about the details of unit testing, the philosophy behind it and about the various tools available, please visit the project pages of QtTest, JUnit and CPPUNIT. The following explanation describes what unit testing is:

For each class developed in a project, an accompanying test container is developed when the interface of the class is defined but before the implementation of the class starts. The test container consists out of testcases that perform all necessary tests on the class while verifying the results. One or more of these test containers (for more than one class) form a test suite.

Your might think, that it is strange to first define the interface, then develop the tests and then start the development of the actual code, but it has shown, that this approach has a couple of interesting side-effects:

- The developer spends time to think about how to test his implementation before he actually works on the implementation. This leads to the fact, that while working on the implementation, he already knows how his code will be tested.

- A clear definition of the end of implementation exists due to the fact, that the testcases will all fail before the beginning of the implementation phase. Once implementation proceeds, more and more testcases will pass. When they all pass, the development is finished.

- Since the tests will run automated and will be re-run very often during the development cycle, a lot of problems will be caught very early on. This reduces the number of problems found during integration of the project. Believe me, there will be plenty left!

Now, the list of all these side-effects is actually the answer to the question Why unit testing? or does anyone have a argument against it? I agree, that in some cases automated unit testing is hard to achieve (e.g. for GUI related code) but I found, that whenever it is possible to introduce automated unit tests, the benefit is huge.

## Unit testing in &app;

Just about the time, when the &app; project underwent a radical change of it's inner business logic (the KMyMoney engine), I read an article about the existance of a unit test container for C++ projects named CPPUNIT. In discussions with my colleagues at work, I got the impression, that this would be something worth to look into. So I sat down and wrote the first test cases for existing code to get a feeling for what is required.

When &app; was ported to KDE4 the CPPUNIT test engine was replaced with the QtTest test engine. We did this because &app; is a KDE, thus a Qt, application so it is better to use Qt's unit test engine than to have an extra dependency on CPPUNIT.

I found it annoying to write test cases for code that existed and was believed to work (version 0.4 of the project). When the decission was made to start with the 0.5 development branch, I started working on the new engine code that should introduce a clear interface between the business logic and the user interface. Another design goal was to write the engine in such a way, that it is not based on any KDE code which the old one was. The second goal to free it from Qt based code was not that easy and was skipped by the project team at that time.

Even if it was hard for me at first to follow the above laid out principle to design the interface, write the test code and then start with the implementation, I followed this trail. It has proven to be very valuable. Once the interface was designed, I started thinking in a different manner: How can I get this class to fail? What strange things could I do to the code from the outside? Those were the design drivers for the test code. And in fact, this thinking changed the way I actually implemented the code, as I knew there was something that would check all these things over and over again automatically.

A lot of code was implemented and when I was almost done with the first shot of the implementation, discussion came up on the developers mailing list about a feature called double entry accounting that was requested for &app; by a few people. The engine I wrote up to that point in time did not cover the aspects of double entry accounting at all, though a few things matched. After some time of discussions, we became a better understanding of the matter and I changed the code to cover double entry accounting. Some of the classes remained as they were, others had to be adopted and yet others rewritten entirely. The testcode had to be changed as well due to the change in the interfaces, but not the logic of the tests. Most of the thoughts how to uncover flaws remained.

And that is another reason, why unit testing is so useful: You can change your internal implementation and still get a feeling, if your code is working or not. And believe me: even if some changes are small, one usually oversees a little side-effect here and there. If one has good unit tests this is not a problem anymore, as those side-effects will be uncovered and tested.

During the course of implementing the engine, I wrote more than 100 testcases. Each testcase sets up a testenvironment for the class and tests various parameters against the class' methods in this environment in so called test steps. Exceptions are also tested to be thrown. The testcases handle unexpected exceptions as well as expected exceptions that do not occur.

## Unit testing HOWTO

This section of the developer handbook should give some examples on how to organize test cases and how to setup a test environment.

My examples will all be based on the code of the &app; engine found in the subdirectory `kmymoney/kmymoney/mymoney` and it's subdirectory `storage`.

### Integration of QtTest into the &app; project

The &app; build system has been configured to build the unit tests only when the KDE4_BUILD_TESTS cmake variable is set to ON. To do this pass the -DKDE4_BUILD_TESTS=ON option while configuring the project using cmake.

In order to add a unit test source file to the build system the following cmake macros should be used:

    IF( KDE4_BUILD_TESTS )

      # UnitTestName
      SET(unittestname_SRCS unittestname.cpp)
      KDE4_ADD_UNIT_TEST(unittestname TESTNAME kmymoney-unittestname ${unittestname_SRCS})
      ADD_DEPENDENCIES(unittestname make-targets-upon-which-the-unit-test-depends)
      TARGET_LINK_LIBRARIES(unittestname libraries-upon-which-the-unit-test-depends)

    ENDIF( KDE4_BUILD_TESTS )

For an example see `kmymoney/mymoney/CmakeLists.txt`.

### Naming conventions

The test containers are also classes. For a given class MyMoneyAbc, which resides in the files `mymoneyabc.h` and `mymoneyabc.cpp`, the test container is named MyMoneyAbcTest and resides in the files `mymoneyabctest.h` and `mymoneyabctest.cpp` in the same directory. The test container must be derived publicly from `QObject` and it mus contain the `Q_OBJECT` macro. Each testcase is given a descriptive name (e.g. EmptyConstructor) and I found it useful to prefix this name with the literal 'test' resulting into something like testEmptyConstructor.

### Necessary include files

In order to use the functionality provided by QtTest, one has to include some information provided with QtTest in the test environment. This is done with the following include primitive as one of the first things in the header file of the test case container (e.g. mymoneyabctest.h):

    #include <QtTest/QtTest>

### Accessing private members

For the verification process it is sometimes necessary to look at some internal states of the object under test. Usually, all this information is declared private in the class and only accessible through setter and getter methods. Cases exist, where these methods are not implemented on purpose and thus accessing the information from the test container is not possible.

Various mechanism have been developed all with pros and cons. The old method was by redefining the specifier private through public but only for the time when reading the header file of the object under test. This leads to a problem when the linker includes the access specifiers in the mangled C++ function name. Because in the library where the tested object resided one of it's functions had one name and in the unit test it had another (due to the different access specifiers) a link error would occur.

The current method consists in having the KMM_MYMONEY_UNIT_TESTABLE macro placed in the declaration of each class that has a unit test. KMM_MYMONEY_UNIT_TESTABLE is usually expanded to nothing but when the header of an object is included in a unit test KMM_MYMONEY_UNIT_TESTABLE is expanded to the following statement friend class UnitTestClassName;. This definition allows the unit test class to access all members of the tested class and does not cause the linkage error caused by the older method.

    #define KMM_MYMONEY_UNIT_TESTABLE friend class MyMoneyAbcTest;
    #include "mymoneyabc.h"

### Standard methods for each testcase

The init() and initTestCase() slots will be called, if they exist, before the execution of any test case. cleanup() and cleanupTestCase() will be called after the execution of the test case, no matter if the test case passes or fails. Thus init() is used to perform initialization necessary for each test case in the unit test class and cleanup() is used to clean things up. init() and cleanup() should be written in such a way, that all objects created through a test case should be removed by cleanup(), i.e. the environment is restored exactly to the state it was before the call to init().

This is not always the case within the testcase for &app;. Espacially when using a database as the permanent storage things have to be overhauled for e.g. MyMoneyFileTest.

QtTest comes with a set of macros that help writing testcases. I cover them here briefly. If you wish a more detailed description, please visit the QtTest page.

### QVERIFY

This is the macro used at most throughout the test cases. It checks, that a given assumption is true. If it is not, the test case fails and a respective message will be printed at the end of the testrun.

QVERIFY has a single argument which is a boolean expression. The expression must be true in order to pass the test. If it is false, the test case fails and no more code of the test case is executed. The following example shows how the macro is used:

      int a, b;
      a = 0, b = 1;
      QVERIFY(a != b);
      a = 1;
      QVERIFY(a == b);

The example shows, how two test steps are combined. One checks the inequality of two integers, one the equality of them. If either one does not work, the test case fails.

See the Unit Test Source File Example for a demonstration of it's use.

### QFAIL

This is the macro used when the execution of a test case reaches a point it should not. This usually happens, if exceptions are thrown or not thrown.

CPPUNIT_FAIL has a single argument which is the error message to be displayed. The following example shows how the macro is used:

      int a = 1, b = 0;
      try {
        a = a / b;
        QFAIL("Expected exception missing!");
      } catch (const exception &e) {
      }

      try {
        a = a / a;
      } catch (const exception &e) {
        QFAIL("Unexpected exception!");
      }

The example shows, how two test steps are combined. One checks the occurance of an exception, the other one that no exception is thrown. If either one does not work, the test case fails.

### QTEST_MAIN

This macro is used as the first thing in the declaration of the test fixture. A single argument is the name of the class for the test fixture. It starts the list of test cases defined as slots of the unit test class.

See the Unit Test Header File Example for a demonstration of it's use.