- Qt 6.9
- Qt Test
- Qt Test Tutorial
- Chapter 5: Writing a Benchmark
- Qt 6.9.3 Reference Documentation

Chapter 4: Replaying GUI Events ◦ Chapter 6: Skipping Tests with QSKIP

# Chapter 5: Writing a Benchmark

This chapter demonstrates how to write benchmarks using Qt Test.

## Writing a Benchmark

To create a benchmark we extend a test function with a QBENCHMARK macro. A benchmark test function will then typically consist of setup code and a QBENCHMARK macro that contains the code to be measured. This test function benchmarks QString::localeAwareCompare().

     void TestBenchmark::simple()
     {
         QString str1 = QLatin1String("This is a test string");
         QString str2 = QLatin1String("This is a test string");

         QCOMPARE(str1.localeAwareCompare(str2), 0);

         QBENCHMARK {
             str1.localeAwareCompare(str2);
         }
     }

Setup can be done at the beginning of the function. At this point, the clock is not running. The code inside the QBENCHMARK macro will be measured, and possibly repeated several times in order to get an accurate measurement.

Several back-ends are available and can be selected on the command line.

## Data Functions

Data functions are useful for creating benchmarks that compare multiple data inputs, for example locale aware compare against standard compare.

     void TestBenchmark::multiple_data()
     {
         QTest::addColumn<bool>("useLocaleCompare");
         QTest::newRow("locale-aware-compare") << true;
         QTest::newRow("standard-compare") << false;
     }

The test function then uses the data to determine what to benchmark.

     void TestBenchmark::multiple()
     {
         QFETCH(bool, useLocaleCompare);
         QString str1 = QLatin1String("This is a test string");
         QString str2 = QLatin1String("This is a test string");

         int result;
         if (useLocaleCompare) {
             QBENCHMARK {
                 result = str1.localeAwareCompare(str2);
             }
         } else {
             QBENCHMARK {
                 result = (str1 == str2);
             }
         }
         Q_UNUSED(result);
     }

The `if (useLocaleCompare)` switch is placed outside the QBENCHMARK macro to avoid measuring its overhead. Each benchmark test function can have one active QBENCHMARK macro.

## Building the Executable

You can build the test case executable using CMake or qmake.

### Building with CMake

Configure your build settings in your CMakeLists.txt file:

     # Copyright (C) 2022 The Qt Company Ltd.
     # SPDX-License-Identifier: LicenseRef-Qt-Commercial OR BSD-3-Clause

     cmake_minimum_required(VERSION 3.16)
     project(tutorial5 LANGUAGES CXX)

     find_package(Qt6 REQUIRED COMPONENTS Core Gui Test Widgets)

     qt_standard_project_setup()

     qt_add_executable(tutorial5
         benchmarking.cpp
     )

     set_target_properties(tutorial5 PROPERTIES
         WIN32_EXECUTABLE TRUE
         MACOSX_BUNDLE TRUE
     )

     target_link_libraries(tutorial5 PRIVATE
         Qt6::Core
         Qt6::Gui
         Qt6::Test
         Qt6::Widgets
     )

     install(TARGETS tutorial5
         BUNDLE  DESTINATION .
         RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR}
         LIBRARY DESTINATION ${CMAKE_INSTALL_LIBDIR}
     )

     qt_generate_deploy_app_script(
         TARGET tutorial5
         OUTPUT_SCRIPT deploy_script
         NO_UNSUPPORTED_PLATFORM_ERROR
     )
     install(SCRIPT ${deploy_script})

Next, from the command line, run either `cmake` or use the `qt-cmake` convenience script located in `Qt-prefix/<version>/<platform>/bin/qt-cmake`:

     <Qt-prefix>/<version>/<platform>/bin/qt-cmake <source-dir> <build-dir> -G Ninja

Then, run your preferred generator tool to build the executable. Here, we're using Ninja:

     ninja

### Building with qmake

Configure your build settings in your `.pro` file:

     QT += widgets testlib

     SOURCES = benchmarking.cpp

     # install
     target.path = $$[QT_INSTALL_EXAMPLES]/qtestlib/tutorial5
     INSTALLS += target

Next, run `qmake`, and, finally, run `make` to build your executable:

     qmake
     make

## Running the Executable

Running the resulting executable should give you the following output:

     ********* Start testing of TestBenchmark *********
     Config: Using QtTest library %VERSION%, Qt %VERSION%
     PASS   : TestBenchmark::initTestCase()
     PASS   : TestBenchmark::simple()
     RESULT : TestBenchmark::simple():
          0.00030 msecs per iteration (total: 79, iterations: 262144)
     PASS   : TestBenchmark::multiple(locale-aware-compare)
     RESULT : TestBenchmark::multiple():"locale-aware-compare":
          0.00029 msecs per iteration (total: 78, iterations: 262144)
     .....
     PASS   : TestBenchmark::series(locale-aware-compare:8001)
     RESULT : TestBenchmark::series():"locale-aware-compare:8001":
          0.039 msecs per iteration (total: 81, iterations: 2048)
     Totals: 15 passed, 0 failed, 0 skipped, 0 blacklisted, 3971ms
     ********* Finished testing of TestBenchmark *********