Dear Readers!

Today I’d like to share a solution to a problem I encountered while using the QtTest framework, namely trying to launch multiple gui test classes from within the same project executable. As I was researching the topic, I realized that Qt Tests were never intended to be run from a single central place, instead the Qt Documentation suggests creating an executable for each test class you have. Let’s see an example:

C++

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
// myguiclasstest.cpp

#include "MyGuiClass.h"
#include <QtTest>

class MyGuiClassTest : public QObject {
    Q_OBJECT

private slots:
    void myClassShouldDoCoolStuff();

};

MyGuiClassTest::myClassShouldDoCoolStuff()
{
    // ...
}

QTEST_MAIN(MyGuiClassTest)
#include "myguiclasstest.moc"

The code above is written exactly like the documentation suggests, and as long as you have one class to test it works just fine. But what happens if you need to test another class and you just simply replace everything between #include and QTEST_MAIN(MyClassTest) with another test class? One might reasonably assume, that this shall work just fine, but unfortunately the compiler will not be happy again. The reason for that is, that the QTEST_MAIN macro expands into a main() function (which executes all your test functions inside your test class), and if you have two such macros in your code, it will mean two defined main() functions within the same project.

Obviously this will not work, so one needs to find a way around that macro. It would be also nice to get rid of the final #include with the moc file, which is only needed because the test class is declared and defined in a source file instead of the usual header+source combo. A quick and naive solution for this could be to simply rearrange the file to be a header, but since the main() function can’t be in a header, it will not work. So what could be a nice and elegant solution to all this?

One approach would be to create a runner class, but I found some of such approaches a bit complex for the simple task of executing multiple test classes. So instead of using these I came up with a much simpler approach, which basically consisted of copying out the necessary bits from the QTEST_MAIN macro and placing them into a templated function just like this:

C++

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
// testrunner.h

#ifndef TESTRUNNER_H
#define TESTRUNNER_H

#include <QtTest>

template <typename TestClass>
void runTests(int argc, char* argv[], int* status)
{
    QApplication app(argc, argv);
    app.setAttribute(Qt::AA_Use96Dpi, true);
    QTEST_DISABLE_KEYPAD_NAVIGATION TestClass tc;
    *status |= QTest::qExec(&tc, argc, argv);
}

#endif // TESTRUNNER_H

Compared to some of the other solutions this really is a simple method, and the usage is equally simple:

C++

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
// main.cpp

#include "myguiclasstest.h"
#include "mydialogclasstest.h"

#include "testrunner.h"

int main(int argc, char *argv[])
{

    int status = 0;

    runTests<MyGuiClassTest>(argc, argv, &status);
    runTests<MyDialogClassTest>(argc, argv, &status);

    return status;

}

Since the runner code has been extracted from the macro, we can safely remove QTEST_MAIN from the end of the test file. Please also note that the test classes are now in a header file instead of a source file, so they can now be included normally into main.cpp. This also means that we can get rid of the .moc related #include we talked about at the beginning of this post as well. Finally, we can rearrange the test function code to be inline. Here’s the modified test code:

C++

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
// myguiclasstest.h


#ifndef MYGUICLASSTEST_H
#define MYGUICLASSTEST_H

#include "MyGuiClass.h"
#include <QtTest>

class MyGuiClassTest : public QObject {
    Q_OBJECT

private slots:
    void myClassShouldDoCoolStuff() 
    {
        // ...
    }

};

#endif // MYGUICLASSTEST_H

There are good and bad news about this entire approach. The good news is, that it works just as expected, all tests execute, Qt Creator is able to parse the end result and all tests will end up in the list of the “Test Results” tab. The bad news is, that in the test selection pane of Creator only the test class written last will appear, so you won’t be able to enable/disable most tests from there. There is one other important question here though: will QtTest set up this way work in concert with other test frameworks like Google Test? Why would one want that anyway? Simply because for non-gui tests I find Google Test to be the better choice, so I’d like to create a project, where both test frameworks are used in parallel for different parts of it.

Let’s say I want QtTest for anything gui related, and Google Test for everything else, which means that two separate test projects will be required (becuase both frameworks will need their own test runner executables). Here’s an example project tree:

MyCoolProject
 |--MyCoolProj.pro
 |--MyCoolApp
    |--MyCoolApp.pro
    \--main.cpp
 |--MyGuiLibs
    |--MyGuiLibs.pro
    |--MyGuiClassLib
       |--MyGuiClassLib.pro
       |--mylib1decl.h
       |--mylib1.h
       \--mylib1.cpp
    \--MyDialogClassLib
       |--MyDialogClassLib.pro
       |--mylib2decl.h
       |--mylib2.h
       \--mylib2.cpp
 |--MyLibs
    \--MyCoolLib
       |--MyCoolLib.pro
       |--mylib3.h
       \--mylib3.cpp
 |--GoogleTest
    |--googletest
       |--ci
       |--googlemock
       |--googletest
       |-- ...
       \--WORKSPACE
    |--GoogleTest.pri
    \--GoogleTest.pro
 \--Tests
    |--Tests.pro
    |--GuiTests
       |--GuiTests.pro
       |--main.cpp
       |--myguiclasstest.h
       \--mydialogclasstest.h
    |--NonGuiTests
       |--NonGuiTests.pro
       |--main.cpp
       \--tst_mycoolstuff.cpp
     

Fortunately, this is more than possible to achieve with qmake (with CMake as well), and here’s how:
First we need a subdirs project for all the tests:

qmake

1
2
3
4
TEMPLATE = subdirs

SUBDIRS += NonGuiTests \
           GuiTests

Then we need the project files for the test projects:

qmake

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
// GuiTests.pro

QT += testlib
QT += gui widgets

CONFIG += testcase
CONFIG += qt warn_on depend_includepath
CONFIG += c++14
CONFIG += cmdline

TEMPLATE = app

ROOTPATH = ../..

DEPENDPATH += $$ROOTPATH/MyGuiLibs/MyGuiClassLib \
              $$ROOTPATH/MyGuiLibs/MyDialogClassLib

INCLUDEPATH += $$ROOTPATH/MyGuiLibs/MyGuiClassLib \
               $$ROOTPATH/MyGuiLibs/MyDialogClassLib

LIBS += -L$$ROOTPATH/MyGuiLibs/MyGuiClassLib -lMyGuiClassLib \
        -L$$ROOTPATH/MyGuiLibs/MyDialogClassLib -lMyDialogClassLib

SOURCES += \
    main.cpp

HEADERS += \
    myguiclasstest.h \
    mydialogclasstest.h \
    testrunner.h \

qmake

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
// NonGuiTests.pro

TARGET = NonGuiTests
TEMPLATE = app

CONFIG += c++14
CONFIG += testcase #Creates 'check' target in Makefile.
CONFIG -= debug_and_release
CONFIG += cmdline

ROOTPATH = ../..

INCLUDEPATH += $$ROOTPATH/MyLibs/MyCoolStuffLib

LIBS += -L$$ROOTPATH/MyLibs/MyCoolStuffLib -lMyCoolStuffLib

include($$ROOTPATH/GoogleTest/GoogleTest.pri)

HEADERS += \


SOURCES += main.cpp \
    tst_mycoolstuff.cpp \

This setup will allow to run everything as inteded without too much hastle. My only problem is the aforementioned lack of test disabling capabilities from within Creator, but sorting that out will have to be postponed for later.

As always, thanks for reading.

UPDATE:

After using this solution a bit with Creator it became clear, that there is something strange about Creator’s detection of tests, as after closing and then later openening Creator, none of the tests were detected. As it turns out, if a list of tests gets created during a session, the list doesn’t get saved in any way for later sessions. Instead, the list is rebuilt at every opening of the project. This sort of behavior wouldn’t be a problem in itself, but unfortunately rebuilding a test list from scratch and gradually changing it during a session doesn’t produce the same results in Creator.

I tried to circumvent this problem by replacing the templated function mentioned earlier in this post with the following function-like macro:

C++

1
2
3
4
5
6
7
#define RUN_TESTS(test_class, argc, argv, status)       \
    {                                                   \
        QApplication app(argc, argv);                   \
        app.setAttribute(Qt::AA_Use96Dpi, true);        \
        QTEST_DISABLE_KEYPAD_NAVIGATION test_class tc;  \
        *status |= QTest::qExec(&tc, argc, argv);       \
    }

Since macros get processed one step earlier in the entirety of the compilation process than templates, I thought Creator will be able to parse the tests better. Unfortunately I was wrong. After closing and opening Creator again, all the tests were gone from the list once more. The biggest problem here is, that unless at least one test is detected, none of the tests will be launched. As such, in order to at least be able to launch all tests, one needs to insert at least one expanded macro into the main function. From that point it actually doesn’t matter if the macro or templated function is used for launching the rest of the tests, both of them seem to work. This is a very ugly workaround indeed, but for now I can’t seem to find another way to launch all the tests, and to be frank, I haven’t had the time to check how exactly Creator parses the tests. This will have to do for now…