Unit Testing

Catkin can run C++ unit tests using the googletest library in C++ and the nosetest utility in Python. This page includes instructions for running tests, writing C++ unit tests and writing Python unit tests.

Running Tests with Catkin

Catkin can run all tests in the workspace by running the commands:

$ catkin build --make-args tests
$ catkin build --make-args test

The first command (the tests target) builds the tests and the second command runs them. You can safely omit the first command if you are only running unit tests on Python packages, which do not need to be built.

Additionally, you can restrict testing to a list of Catkin packages:

$ catkin build --make-args tests -- my_package1 my_package2
$ catkin build --make-args test -- my_package1 my_package2

This command produces no output. You can use the catkin_test_results tool to view the test results:

$ catkin_test_results build
Summary: 32 tests, 0 errors, 0 failures

Manually Running a Test

It's often useful to manually run tests, completely bypassing Catkin, during development. This lets you to see which tests failed, view their console output, and run specific test suites. This is always possible because Catkin simply invokes googletest and nosetest.

Manually Running a Test (C++)

Every catkin_add_gtest CMake command builds an executable. You can manually run a test by invoking that executable. For example, if you have the entry catkin_add_gtest(my_test test/my_test.cpp) in the my_package package, you can run:

$ rosrun my_package my_test

googletest supports a number of command line options for controlling which test to run, how many times to run them, and how to format the test results. You can see the full list of options by passing the --help flag. One of the most useful options is to filter which tests to run. First, view a list of tests by passing the --gtest_list_tests flag.

$ rosrun my_package my_test --gtest_list_tests
MyFixture.
test1
test2
OtherFixture
test3

You can pass any of those names to --gtest_filter option to explicitly enable (or disable) a particular test:

$ rosrun my_package my_test --gtest_filter=MyFixture.test1 # only test1
$ rosrun my_package my_test --gtest_filter=-MyFixture.test1 # everything but test1
$ rosrun my_package my_test --gtest_filter=MyFixture.* # only tests in MyFixture

Manually Running a Test (Python)

You can use the nosetests command to directly run your Python unit tests:

$ nosetests -v prpy/tests/planning # run tests in a directory
$ nosetests -v prpy/tests/planning/test_CBiRRT.py # run tests in a file

The -v flag tells nosetests to print the name of each test before it runs it. You can run a specific test by passing appending its name to the end of the path using the syntax path/to/test.py:MyTestSuite.test_MyTestName. You may also want to pass the -s flag, which tells nosetests to print the output of the test immediately. For example, you could run:

$ nosetests -s prpy/tests/planning/test_CBiRRT.py:CBiRRTPlannerTest.test_PlanToConfiguration_GoalInCollision_Throws

You can view the full list of tests, without running them, by running

$ nosetests -v --collect-only prpy/tests/planning

Writing Tests

Test Dependencies

You should specify any dependencies that are required for unit tests using <test_depend> tags in your package's package.xml file. At a minimum, C++ packages typically depend on gtest and Python packages typically depend on python-nose. All <build_depend> and <exec_depend> are implicitly considered test dependencies and, thus, should not be duplicated.

Writing Tests (C++)

You should use the Google Test framework to write C++ unit tests. The primer and samples pages in the Google Test documentation do a good job explaining the basics. The advanced guide and reference sheet outline the full set of EXPECT and ASSERT macros available for use.

Once you written your tests, you can register them with Catkin using the catkin_add_gtest macro:

if(CATKIN_ENABLE_TESTING)
    catkin_add_gtest(test_MyClass1 tests/test_MyClass1.cpp)
    catkin_add_gtest(test_MyClass2 tests/test_MyClass2.cpp)
endif(CATKIN_ENABLE_TESTING)

This command defines a new target. Remember that you must set target_link_libraries and other target-specific attributes that are necessary to build the test. In general, you can treat catkin_add_gtest like any other add_executable command in your CMakeLists.txt file.

Writing Tests (Python)

import os.path, sys
sys.path = [os.path.abspath(os.path.dirname(__file__))] + sys.path

You should use the unittest module, which is part of the standard library, to write unit tests in Python. Tests are typically stored in the tests/ directory, are prefixed by test_ and do not have the executable bit set.

Each file or directory of tests must be registered with Catkin in the CMakeLists.txt file using the catkin_add_nosetests function if the CATKIN_ENABLE_TESTING variable is set. It's personal preference whether to put all tests in one file or split them among multiple files; use your best judgement.

Suppose we are writing tests for the my_package package and they are split between tests/test_MyClass1.py and tests/test_MyClass2.py. The CMakeLists.txt file should include the block:

if(CATKIN_ENABLE_TESTING)
    catkin_add_nosetests(tests/test_MyClass1.py)
    catkin_add_nosetests(tests/test_MyClass2.py)
endif(CATKIN_ENABLE_TESTING)

Alternatively, you can register the entire tests directory with nosetests:

if(CATKIN_ENABLE_TESTING)
    catkin_add_nosetests(tests)
endif(CATKIN_ENABLE_TESTING)