For building a reliable project, we need to set up tests to check if the features work correctly with various inputs. CMake comes with a few tools for testing, which we will cover in this article.

Setting Up

Typically, we set up a tests directory, which stores the test subdirectory for each feature we want to test. Each test subdirectory should contain source code for the test, typically named tester, and a CMakeLists.txt file for building the test executable. Below is the file structure of the example we will use, featuring a custom Graph library that implements an adjacency list and graph search algorithms, BFS and DFS. The goal is to test if BFS and DFS work as expected.

CMakeLists.txt
main.cc
Graph
tests/
├── CMakeLists.txt
├── bfs/
│   ├── CMakeLists.txt
│   └── tester.cc
├── dfs/
│   ├── CMakeLists.txt
│   └── tester.cc

In the top-level CMakeLists.txt, we can define options to test BFS and/or DFS and decide whether to include the tests subdirectory or just build the executable main.cc, as shown below.

cmake_minimum_required(VERSION 3.31)
 
project(Project VERSION 1.0)
 
option(TEST_BFS "Testing BFS" ON)
option(TEST_DFS "Testing DFS" ON)
 
add_subdirectory(Graph)
 
if(TEST_BFS OR TEST_DFS)
    add_subdirectory(tests)
endif()
 
if(NOT (TEST_BFS OR TEST_DFS))
    add_executable(${PROJECT_NAME} main.cc)
 
    target_link_libraries(${PROJECT_NAME} Graph)
 
    target_include_directories(${PROJECT_NAME} PRIVATE ${CMAKE_SOURCE_DIR}/Graph/include)
endif()

Since the Graph library is expected to be used in both tests and the main source code, we include the Graph subdirectory before checking the options. This setup allows us to isolate the logic related to testing from the other directories.

CTest

CMake provides the testing program CTest, which allows us to set up and execute tests easily. To use it, we need to enable testing by adding enable_testing() to the CMakeLists.txt file in the tests directory and set up test cases using add_test, as follows.

cmake_minimum_required(VERSION 3.31)
enable_testing()
 
if(TEST_BFS)
    add_subdirectory(bfs)
    add_test(NAME "BFS_TEST1" COMMAND bfs_test 2 6 0 WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR})
    add_test(NAME "BFS_TEST2" COMMAND bfs_test 0 4 1 WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR})
endif()
 
if(TEST_DFS)
    add_subdirectory(dfs)
    add_test(NAME "DFS_TEST1" COMMAND dfs_test 2 6 0 WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR})
    add_test(NAME "DFS_TEST2" COMMAND dfs_test 0 4 1 WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR})
endif()

Here, bfs_test and dfs_test are the names of the executables, which are supposed to receive the start node, target node, and whether there is a path from the start node. Both BFS and DFS are expected to correctly identify whether the target node is reachable from the start node. If a test returns 0, CTest classifies it as a success; otherwise, it is considered a failure. Below is an example of tester.cc for BFS, compiled to bfs_test.

#include <AdjacencyList.h>
#include <GraphSearch.h>
#include <stdlib.h>
 
using namespace GraphSearch;
 
int main(int argc, char* argv[]) {
 
    AdjacencyList g1(5);
    g1.addEdge(0, 1);
    g1.addEdge(0, 2);
    g1.addEdge(1, 3);
    g1.addEdge(2, 4);
 
    int node1 = atoi(argv[1]);
    int node2 = atoi(argv[2]);
    int expectedResult = atoi(argv[3]);
 
    return (expectedResult == BFS(g1, node1, node2));
}

We use atoi from <stdlib.h>, which converts characters into integers. The above tester.cc can be built with a standard CMakeLists.txt file by linking the necessary Graph library and specifying the location of the include directory. We can build the project using cmake -S . -B build/ and make in the build directory, which will create a tests directory within the build directory.

When we navigate to the build/tests directory and run ctest -N, we should see a list of all the tests we have set up, which can be executed with ctest -VV. The above test should succeed if the correct implementations are provided in Graph. We can easily disable tests by adding -DTEST_BFS=OFF and -DTEST_DFS=OFF when invoking cmake and run ./Project within the build directory.

CDash

We can use CTest to verify that features work in the development environment, but we also want to track test results from others running the tests in different environments to ensure the features are correctly set up. CDash is an open-source project that provides a dashboard for this purpose. We can either set up our own CDash server or use their website.

After registering an account on their dashboard, we can create a new project by providing the necessary information, such as the project name, description, and repository locations. Once the project is created, we can navigate to the project section and then to the miscellaneous section to find CTestConfig.cmake, which we can copy to create a CTestConfig.cmake file in our project directory.

After pushing changes to the remote repository, we include include(CTest) after adding the tests subdirectory in the top-level CMakeLists.txt file and build the project. Then, we run ctest -D Experimental under build/tests and check the dashboard to see the updated test results. If you are interested, I recommend watching the CDash video cited at the bottom of the article and reviewing the official documentation.

Conclusion

In this article, we covered how to set up and run tests using CTest and briefly introduced CDash for setting up a dashboard. CDash requires a remote repository, which we will cover in a future article. I hope the last few articles on CMake have helped you understand the basics of CMake for building projects.

Resources

• Code, Tech, and Tutorials. 2023. CMake Tutorial EP 13 | CTest. YouTube.
• Code, Tech, and Tutorials. 2024. CMake Tutorial EP 14 | CDash. YouTube.