# Quickstart with CMake This tutorial describes how to set up your development environment with CMake as the build system, and how to get a simple fuzz test up and running. We recommend this tutorial if you're new to FuzzTest or if you need a quick refresher. ## Definitions The library can work in several different modes. * Unit test mode (*default*): FUZZ_TESTs are built without coverage instrumentation and run with random inputs for short period of time along with regular unit TESTs. * Fuzzing mode (controlled by `-DFUZZTEST_FUZZING_MODE=on`): FUZZ_TESTs are built with coverage instrumentation and run individually and indefinitely or for specified amount of time. * Compatibility mode (controlled by `-DFUZZTEST_COMPATIBILITY_MODE=libfuzzer`): like Fuzzing mode, but an external engine is used instead FuzzTest's built in engine. Currently [libFuzzer](https://llvm.org/docs/LibFuzzer.html) is supported. **Warning:** compatibility mode is experimental and does not guarantee full FuzzTest features. ## Overall rules We provide a default CMake option for unittest mode, the instructions are straightforward and work like [googletest](https://google.github.io/googletest/quickstart-cmake.html). To run in a fuzztest mode, you need to make sure all targets are compiled in a special way. * `-g -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION -UNDEBUG` * `-fsanitize-coverage=inline-8bit-counters -fsanitize-coverage=trace-cmp` * `-fsanitize=address -DADDRESS_SANITIZER` (optionally enabling sanitizer) The FuzzTest framework doesn't need to be built with these flags, only the target project. For all projects to have correct flags set, use a CMake macro `fuzztest_setup_fuzzing_flags()`. For a concrete example, see the root CMakeLists.txt. First, we build framework without additional flags and then build the rest of translation units with coverage flags. Otherwise we might get into recursive issues, i.e. fuzztest framework will be fuzzing itself. Easiest way to make sure you build everything correctly is to invoke `add_subdirectory` first in your project on fuzztest. ## Prerequisites To use FuzzTest, you'll need: * A Linux-based operating system. * [Clang](https://clang.llvm.org/). GCC is not yet supported. Unittest mode mode should work but it's not yet tested. * [CMake](https://https://cmake.org/). ## Set up a CMake workspace First, create a directory where your project will reside: ```sh $ mkdir first_fuzz_project && cd first_fuzz_project $ git clone https://github.com/google/fuzztest.git ``` Create a `CMakeLists.txt`. ``` cmake_minimum_required(VERSION 3.19) project(first_fuzz_project) # GoogleTest requires at least C++17 set(CMAKE_CXX_STANDARD 17) add_subdirectory(fuzztest) ``` FuzzTest CMake already handles its dependencies on its own: [RE2](https://github.com/google/re2), [Abseil](https://github.com/abseil/abseil-cpp). We highly recommend using [GoogleTest](https://github.com/google/googletest) as well. ## Create and run a fuzz test With the CMake workspace set up, you can start using FuzzTest. Let's create a trivial example to make sure everything runs correctly. Create a file named `first_fuzz_test.cc` in the directory `first_fuzz` with the following contents: ```c++ #include "fuzztest/fuzztest.h" #include "gtest/gtest.h" TEST(MyTestSuite, OnePlustTwoIsTwoPlusOne) { EXPECT_EQ(1 + 2, 2 + 1); } void IntegerAdditionCommutes(int a, int b) { EXPECT_EQ(a + b, b + a); } FUZZ_TEST(MyTestSuite, IntegerAdditionCommutes); ``` The file contains two tests in the test suite named `MyTestSuite`. The first one is a unit test named `OnePlustTwoIsTwoPlusOne` asserting that integer addition commutes for two specific values. The second test is a fuzz test that captures the commutative property of integer addition more generally. The test consists of a property function with a suggestive name `IntegerAdditionCommutes`, and the test registration using the macro `FUZZ_TEST`. The property function asserts the commutative property for two generic integer parameters. To build and run the tests, **add** to the file `CMakeLists.txt` the following content: ```cmake ... enable_testing() include(GoogleTest) add_executable( first_fuzz_test first_fuzz_test.cc ) # If you have other dependencies than FuzzTest, link them: # target_link_libraries(first_fuzz_test PRIVATE other_deps) link_fuzztest(first_fuzz_test) gtest_discover_tests(first_fuzz_test) ``` This defines a C++ test binary and links it with the FuzzTest and GoogleTest libraries, as well as the version of the default GoogleTest main that supports FuzzTest. ### Unit test mode There are several ways to run the tests: in the unit test mode and in the fuzzing mode (see above). The unit test mode runs both test for a short time without sanitizer and coverage instrumentation: ``` $ mkdir build && cd build $ CC=clang CXX=clang++ cmake \ -DCMAKE_BUILD_TYPE=RelWithDebug .. $ cmake --build . $ ./first_fuzz_test [==========] Running 2 tests from 1 test suite. [----------] Global test environment set-up. [----------] 2 tests from MyTestSuite [ RUN ] MyTestSuite.OnePlustTwoIsTwoPlusOne [ OK ] MyTestSuite.OnePlustTwoIsTwoPlusOne (0 ms) [ RUN ] MyTestSuite.IntegerAdditionCommutes [ OK ] MyTestSuite.IntegerAdditionCommutes (13 ms) [----------] 2 tests from MyTestSuite (13 ms total) [----------] Global test environment tear-down [==========] 2 tests from 1 test suite ran. (13 ms total) [ PASSED ] 2 tests. ``` ### Fuzzing mode The fuzzing mode runs a single specified fuzz test with sanitizer and coverage instrumentation. It keeps running the test with different input values until it finds a crash or it is explicitly terminated by the user. Before `add_executable(first_fuzz_test ...)`, add the specified flags: ```cmake fuzztest_setup_fuzzing_flags() ``` Then run: ``` $ CC=clang CXX=clang++ cmake \ -DCMAKE_BUILD_TYPE=RelWithDebug \ -DFUZZTEST_FUZZING_MODE=on .. $ cmake --build . $ ./first_fuzz_test --fuzz=MyTestSuite.IntegerAdditionCommutes [.] Sanitizer coverage enabled. Counter map size: 21290, Cmp map size: 262144 Note: Google Test filter = MyTestSuite.IntegerAdditionCommutes [==========] Running 1 test from 1 test suite. [----------] Global test environment set-up. [----------] 1 test from MyTestSuite [ RUN ] MyTestSuite.IntegerAdditionCommutes [*] Corpus size: 1 | Edges covered: 131 | Fuzzing time: 504.798us | Total runs: 1.00e+00 | Runs/secs: 1 [*] Corpus size: 2 | Edges covered: 133 | Fuzzing time: 934.176us | Total runs: 3.00e+00 | Runs/secs: 3 [*] Corpus size: 3 | Edges covered: 134 | Fuzzing time: 2.384383ms | Total runs: 5.30e+01 | Runs/secs: 53 [*] Corpus size: 4 | Edges covered: 137 | Fuzzing time: 2.732274ms | Total runs: 5.40e+01 | Runs/secs: 54 [*] Corpus size: 5 | Edges covered: 137 | Fuzzing time: 7.275553ms | Total runs: 2.48e+02 | Runs/secs: 248 [*] Corpus size: 6 | Edges covered: 137 | Fuzzing time: 21.97155ms | Total runs: 9.12e+02 | Runs/secs: 912 [*] Corpus size: 7 | Edges covered: 137 | Fuzzing time: 166.721522ms | Total runs: 8.49e+03 | Runs/secs: 8491 [*] Corpus size: 8 | Edges covered: 146 | Fuzzing time: 500.398497ms | Total runs: 2.77e+04 | Runs/secs: 27741 [*] Corpus size: 9 | Edges covered: 146 | Fuzzing time: 500.821386ms | Total runs: 2.77e+04 | Runs/secs: 27742 [*] Corpus size: 10 | Edges covered: 147 | Fuzzing time: 2.597553524s | Total runs: 1.75e+05 | Runs/secs: 87476 ^C ``` Congratulations! You're now all set for fuzzing with FuzzTest. ### Compatibility mode **Warning:** compatibility mode is experimental and does not guarantee full FuzzTest features. ``` $ CC=clang CXX=clang++ cmake \ -DCMAKE_BUILD_TYPE=RelWithDebug \ -DFUZZTEST_COMPATIBILITY_MODE=libfuzzer .. $ cmake --build . $ ./first_fuzz_test --fuzz=MyTestSuite.IntegerAdditionCommutes Note: Google Test filter = MyTestSuite.IntegerAdditionCommutes [==========] Running 1 test from 1 test suite. [----------] Global test environment set-up. [----------] 1 test from MyTestSuite [ RUN ] MyTestSuite.IntegerAdditionCommutes FUZZTEST_PRNG_SEED=OffQXb8u5_vZtH4-7wgVOLu_HNAhPIbLz7CFF13u3nk INFO: found LLVMFuzzerCustomMutator (0x55edfe61cac0). Disabling -len_control by default. INFO: libFuzzer ignores flags that start with '--' INFO: Running with entropic power schedule (0xFF, 100). INFO: Seed: 1109364873 INFO: Loaded 1 modules (18692 inline 8-bit counters): 18692 [0x55edfe975548, 0x55edfe979e4c), INFO: Loaded 1 PC tables (18692 PCs): 18692 [0x55edfe979e50,0x55edfe9c2e90), INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 4096 bytes INFO: A corpus is not provided, starting from an empty corpus #2 INITED cov: 17 ft: 18 corp: 1/1b exec/s: 0 rss: 38Mb #3 NEW cov: 104 ft: 108 corp: 2/66b lim: 4096 exec/s: 0 rss: 38Mb L: 65/65 MS: 1 Custom- #4 NEW cov: 105 ft: 112 corp: 3/121b lim: 4096 exec/s: 0 rss: 38Mb L: 55/65 MS: 1 Custom- #5 NEW cov: 105 ft: 113 corp: 4/196b lim: 4096 exec/s: 0 rss: 38Mb L: 75/75 MS: 1 Custom- #16 REDUCE cov: 105 ft: 113 corp: 4/195b lim: 4096 exec/s: 0 rss: 38Mb L: 54/75 MS: 1 Custom- #41 NEW cov: 105 ft: 114 corp: 5/247b lim: 4096 exec/s: 0 rss: 38Mb L: 52/75 MS: 5 Custom-Custom-Custom-Custom-Custom- #65 NEW cov: 106 ft: 115 corp: 6/312b lim: 4096 exec/s: 0 rss: 38Mb L: 65/75 MS: 4 Custom-Custom-Custom-Custom- #69 NEW cov: 108 ft: 119 corp: 7/365b lim: 4096 exec/s: 0 rss: 38Mb L: 53/75 MS: 4 Custom-Custom-Custom-Custom- ``` ## Next steps * This tutorial covered the basic setup of the build environment with CMake. To learn more about the FuzzTest framework and how to use it, check out the [Codelab](tutorial.md). * Explore the rest of the [documentation](./).