Running Unit Tests
Unit tests are small tests of individual “units” of code. They allow developers and users to check that specific parts of the code are behaving as expected. These tests differ from integration or validation tests, which test how different units of the code interact.
If you are building from source, you can run the unit tests yourself to verify the the specific behaviors in the tests. Unit tests can never be comprehensive; the unit tests may all pass even if there is a bug in the code.
Compiling Unit Tests
Unit tests are only supported with meson builds. You must build from source to build the unit tests. They are not part of the pre-compiled executables. For more information, see Installation, and Build SU2 on Linux/MacOS.
In order to compile the unit tests, add the flag
to your meson configure call. Then, you can build and run the tests by
There are three ways to run the main unit tests:
meson test -C builddir, where
builddiris the build directory.
ninja test -C builddir, where
builddiris the build directory.
./UnitTests/test_driverfrom the SU2 build directory.
If you have run
ninja install, then the
test_driver executable will
also be located in the
bin directory where you have installed SU2. The
first option will call ninja, which will then run the
executable. The second option will call the
The last option, manually running the test driver, gives the most flexibility.
This help page will focus on the command-line options for that last option.
By default, Catch2 will only show the output from failing tests. To also
see the output for failing tests, add the command line argument
The above discussion over-simplifies the test driver setup. There
are actually three test drivers:
test_driver_DD. These test drivers
are built or run depending on the type of installation (e.g. direcdiff,
autodiff). For the most common use-case, you will not have a directdiff
or autodiff build and will only use
test_driver. If you call
ninja test, the correct
drivers will run automatically. For more on tests using algorithmic
differentiation or direct differentiation, see the section “AD and
Direct-differentiation tests” below.
Selecting subsets of the tests
You can also filter tests in two ways. The most basic, top-level grouping is by test cases. You can see all the test cases by running:
You can then select a test case by name. For example, if I want to run the test case “Volume Computation”, I would run:
./UnitTests/test_driver "Volume Computation"
Within each test case, the test sections form trees of arbitrary depth.
Each branch can be selected with the
-c or the
--section command line
argument. For example, if I want to run the test section “2D Edge” from
the test case “Volume Computation”, I would run:
./UnitTests/test_driver "Volume Computation" -c "2D Edge"
Further sub-sections can be selected by chaining together multiple “-c” selections, e.g.
./UnitTests/test_driver "Volume Computation" -c "Section Name" -c "Subsection Name"
You can also filter tests by tags. To see all the available tags, run:
To run tests matching a specific tag, write the tag name in square braces as an argument for the test driver. For example, if I want to run the tests with the tag “Dual Grid”, I would run:
./UnitTests/test_driver "[Dual Grid]"
If you want to run tests matching any of multiple tags, then include them all in the quotations, but keep the square brackets separate. For example:
./UnitTests/test_driver "[Primal Grid], [Dual Grid]"
There is also a fundamental difference between
"[Primal Grid], [Dual Grid]" and
"[Primal Grid][Dual Grid]". The first
selects all test matching either “Primal Grid” or “Dual Grid”.
The second selects all tests matching both “Primal Grid” and “Dual Grid”.
If that’s confusing to you or you would like to know more, then please read
AD and Direct-differentiation tests
While the above discussion focused on the
test_driver, there are also
two additional test drivers for algorithmic differentiation (AD) and
direct-differentiation (DD). Because these executables need to be linked
to different libraries (normal vs AD vs DD libraries), each exists as a
standalone executable. As an example,
test_driver_AD will test the
SU2 libraries compiled with AD support.
Depending on your installation, one or two of these test drivers may be
missing. This is normal. SU2 will only install the test drivers that
match the compilation type you requested. For example, neither
test_driver_DD will be compiled if
you build SU2 with the default options
To run the test drivers manually, you can run any of the three options
from the build directory. If you installed the program, these test
drivers will also be found in the
bin folder of your install directory:
./UnitTests/test_driver ./UnitTests/test_driver_AD ./UnitTests/test_driver_DD
What’s the difference between a test case, a test section, and a tag?
Test cases are the highest level of organization. Each test must belong to exactly one test case. Within a test case, test sections can be used to organize related tests. These test sections form a tree of arbitrary depth. Test sections allow multiple tests to share setup, teardown, or data. However, note that the tests are always run separately from start to finish. Even if two tests use the same data, the two tests cannot interact.
Tags allow you to identify related tests. One important detail is that each test can have multiple tags. The tags can span multiple test sections. Tags allow complicated groupings of tests. Tags do not facilitate sharing of setup, teardown, or data.