mirror of
https://fuchsia.googlesource.com/third_party/pigweed.googlesource.com/pigweed/pigweed
synced 2024-09-20 05:41:06 +00:00
d6577b923d
Change-Id: I943d600443417a61b247b1a5bc51c280866d5ce9 Reviewed-on: https://pigweed-review.googlesource.com/c/pigweed/pigweed/+/116573 Reviewed-by: Anthony DiGirolamo <tonymd@google.com> Commit-Queue: Auto-Submit <auto-submit@pigweed.google.com.iam.gserviceaccount.com> Pigweed-Auto-Submit: Rob Mohr <mohrr@google.com>
196 lines
8.0 KiB
ReStructuredText
196 lines
8.0 KiB
ReStructuredText
.. _module-pw_compilation_testing:
|
|
|
|
======================
|
|
pw_compilation_testing
|
|
======================
|
|
The pw_compilation_testing module provides for negative compilation (NC)
|
|
testing. Negative compilation tests ensure that code that should not compile
|
|
does not compile. Negative compilation testing is helpful in a variety of
|
|
scenarios, for example:
|
|
|
|
- Testing for compiler errors, such as ``[[nodiscard]]`` checks.
|
|
- Testing that a template cannot be instantiated with certain types.
|
|
- Testing that a ``static_assert`` statement is triggered as expected.
|
|
- For a ``constexpr`` function, testing that a ``PW_ASSERT`` is triggered as
|
|
expected.
|
|
|
|
Negative compilation tests are only supported in GN currently. Negative
|
|
compilation tests are not currently supported in GN on Windows due to
|
|
`b/241565082 <https://bugs.pigweed.dev/241565082>`_.
|
|
|
|
.. warning::
|
|
|
|
This module is in an early, experimental state. Do not use it unless you have
|
|
consulted with the Pigweed team.
|
|
|
|
---------------------------------
|
|
Negative compilation test example
|
|
---------------------------------
|
|
.. code-block:: cpp
|
|
|
|
#include "gtest/gtest.h"
|
|
#include "pw_compilation_testing/negative_compilation.h"
|
|
|
|
template <int kValue>
|
|
struct MyStruct {
|
|
static_assert(kValue % 2 == 0, "wrong number!");
|
|
|
|
constexpr int MultiplyOdd(int runtime_value) const {
|
|
PW_ASSERT(runtime_value % 2 == 0);
|
|
return kValue * runtime_value;
|
|
}
|
|
};
|
|
|
|
[[maybe_unused]] MyStruct<16> this_one_works;
|
|
|
|
// NC tests cannot be compiled, so they are created in preprocessor #if or
|
|
// #elif blocks. These NC tests check that a static_assert statement fails if
|
|
// the code is compiled.
|
|
#if PW_NC_TEST(NegativeOddNumber)
|
|
PW_NC_EXPECT("wrong number!");
|
|
[[maybe_unused]] MyStruct<-1> illegal;
|
|
#elif PW_NC_TEST(PositiveOddNumber)
|
|
PW_NC_EXPECT("wrong number!");
|
|
[[maybe_unused]] MyStruct<5> this_is_illegal;
|
|
#endif // PW_NC_TEST
|
|
|
|
struct Foo {
|
|
// Negative compilation tests can go anywhere in a source file.
|
|
#if PW_NC_TEST(IllegalValueAsClassMember)
|
|
PW_NC_EXPECT("wrong number!");
|
|
MyStruct<12> also_illegal;
|
|
#endif // PW_NC_TEST
|
|
};
|
|
|
|
TEST(MyStruct, MultiplyOdd) {
|
|
MyStruct<5> five;
|
|
EXPECT_EQ(five.MultiplyOdd(3), 15);
|
|
|
|
// This NC test checks that a specific PW_ASSERT() fails when expected.
|
|
// This only works in an NC test if the PW_ASSERT() fails while the compiler
|
|
// is executing constexpr code. The test code is used in a constexpr
|
|
// statement to force compile-time evaluation.
|
|
#if PW_NC_TEST(MyStruct_MultiplyOdd_AssertsOnOddNumber)
|
|
[[maybe_unused]] constexpr auto fail = [] {
|
|
PW_NC_EXPECT("PW_ASSERT\(runtime_value % 2 == 0\);");
|
|
MyStruct<3> my_struct;
|
|
return my_struct.MultiplyOdd(4); // Even number, PW_ASSERT should fail.
|
|
}();
|
|
#endif // PW_NC_TEST
|
|
}
|
|
|
|
------------------------------------
|
|
Creating a negative compilation test
|
|
------------------------------------
|
|
- Declare a ``pw_cc_negative_compilation_test()`` GN target or set
|
|
``negative_compilation_test = true`` in a ``pw_test()`` target.
|
|
- Add the test to the build in a toolchain with negative compilation testing
|
|
enabled (``pw_compilation_testing_NEGATIVE_COMPILATION_ENABLED = true``).
|
|
- In the test source files, add
|
|
``#include "pw_compilation_testing/negative_compilation.h"``.
|
|
- Use the ``PW_NC_TEST(TestName)`` macro in a ``#if`` statement.
|
|
- Immediately after the ``PW_NC_TEST(TestName)``, provide one or more
|
|
Python-style regular expressions with the ``PW_NC_EXPECT()`` macro, one per
|
|
line.
|
|
- Execute the tests by running the build.
|
|
|
|
To simplify parsing, all ``PW_NC_TEST()`` statements must fit on a single line.
|
|
``PW_NC_EXPECT()`` statements may span multiple lines, but must contain a single
|
|
regular expression as a string literal. The string may be comprised of multiple
|
|
implicitly concatenated string literals. The ``PW_NC_EXPECT()`` statement cannot
|
|
contain anything else except for ``//``-style comments.
|
|
|
|
Test assertions
|
|
===============
|
|
Negative compilation tests must have at least one assertion about the
|
|
compilation output. The assertion macros must be placed immediately after the
|
|
line with the ``PW_NC_TEST()`` or the test will fail.
|
|
|
|
.. c:macro:: PW_NC_EXPECT(regex_string_literal)
|
|
|
|
When negative compilation tests are run, checks the compilation output for the
|
|
provided regular expression. The argument to the ``PW_NC_EXPECT()`` statement
|
|
must be a string literal. The literal is interpreted character-for-character
|
|
as a Python raw string literal and compiled as a Python `re
|
|
<https://docs.python.org/3/library/re.html>`_ regular expression.
|
|
|
|
For example, ``PW_NC_EXPECT("something (went|has gone) wrong!")`` searches the
|
|
failed compilation output with the Python regular expression
|
|
``re.compile("something (went|has gone) wrong!")``.
|
|
|
|
.. c:macro:: PW_NC_EXPECT_GCC(regex_string_literal)
|
|
|
|
Same as :c:macro:`PW_NC_EXPECT`, but only applies when compiling with GCC.
|
|
|
|
.. c:macro:: PW_NC_EXPECT_CLANG(regex_string_literal)
|
|
|
|
Same as :c:macro:`PW_NC_EXPECT`, but only applies when compiling with Clang.
|
|
|
|
.. admonition:: Test expectation tips
|
|
:class: tip
|
|
|
|
Be as specific as possible, but avoid compiler-specific error text. Try
|
|
matching against the following:
|
|
|
|
- ``static_assert`` messages.
|
|
- Contents of specific failing lines of source code:
|
|
``PW_NC_EXPECT("PW_ASSERT\(!empty\(\));")``.
|
|
- Comments on affected lines: ``PW_NC_EXPECT("// Cannot construct from
|
|
nullptr")``.
|
|
- Function names: ``PW_NC_EXPECT("SomeFunction\(\).*private")``.
|
|
|
|
Do not match against the following:
|
|
|
|
- Source file paths.
|
|
- Source line numbers.
|
|
- Compiler-specific wording of error messages, except when necessary.
|
|
|
|
------
|
|
Design
|
|
------
|
|
The basic flow for negative compilation testing is as follows.
|
|
|
|
- The user defines negative compilation tests in preprocessor ``#if`` blocks
|
|
using the ``PW_NC_TEST()`` and :c:macro:`PW_NC_EXPECT` macros.
|
|
- The build invokes the ``pw_compilation_testing.generator`` script. The
|
|
generator script:
|
|
|
|
- finds ``PW_NC_TEST()`` statements and extracts a list of test cases,
|
|
- finds all associated :c:macro:`PW_NC_EXPECT` statements, and
|
|
- generates build targets for each negative compilation tests,
|
|
passing the test information and expectations to the targets.
|
|
|
|
- The build compiles the test source file with all tests disabled.
|
|
- The build invokes the negative compilation test targets, which run the
|
|
``pw_compilation_testing.runner`` script. The test runner script:
|
|
|
|
- invokes the compiler, setting a preprocessor macro that enables the ``#if``
|
|
block for the test.
|
|
- captures the compilation output, and
|
|
- checks the compilation output for the :c:macro:`PW_NC_EXPECT` expressions.
|
|
|
|
- If compilation failed, and the output matches the test case's
|
|
:c:macro:`PW_NC_EXPECT` expressions, the test passes.
|
|
- If compilation succeeded or the :c:macro:`PW_NC_EXPECT` expressions did not
|
|
match the output, the test fails.
|
|
|
|
Existing frameworks
|
|
===================
|
|
Pigweed's negative compilation tests were inspired by Chromium's `no-compile
|
|
tests <https://www.chromium.org/developers/testing/no-compile-tests/>`_
|
|
tests and a similar framework used internally at Google. Pigweed's negative
|
|
compilation testing framework improves on these systems in a few respects:
|
|
|
|
- Trivial integration with unit tests. Negative compilation tests can easily be
|
|
placed alongside other unit tests instead of in separate files.
|
|
- Safer, more natural macro-based API for test declarations. Other systems use
|
|
``#ifdef`` macro checks to define test cases, which fail silently when there
|
|
are typos. Pigweed's framework uses function-like macros, which provide a
|
|
clean and natural API, catch typos, and ensure the test is integrated with the
|
|
NC test framework.
|
|
- More readable, flexible test assertions. Other frameworks place assertions in
|
|
comments after test names, while Pigweed's framework uses function-like
|
|
macros. Pigweed also supports compiler-specific assertions.
|
|
- Assertions are required. This helps ensure that compilation fails for the
|
|
expected reason and not for an accidental typo or unrelated issue.
|