I figured I’d follow up here since the post weekly meeting discussion was how I was trying, and failing (pun intended) to getting compile-fail tests working with cmake/ctest. I was at the point of thinking there might be a ctest bug. Anyway, of course there isn’t – user error in the end. ctest –verbose turns out to be super important to see what’s really. happeing. As it turns out my issue was failing to add the library dependency so the header path would be in the compile.
This isn’t yet for a Beman library – although I have uses, but I wrote it as a small cmake function that we could pull in here. @vito.gamberini will no doubt have a critique of it. One thing that’s a bit of a pain is that the number of test files explodes quickly. The only way I can think to solve that is to have one source file and have macro flags for various tests – ugly enough that I’d prefer to just have lots of files.
They could but it’s not needed in this case. That is, the compiler returns a result of 1 which is a fail code for ctest. Note I tried regex and it didn’t work for me – not sure what the syntax is. But I’ll admit when I realized I didn’t need it I did not really pursue….
add_executable creates build rules for compiling and linking. Since you only care about the build rule for compilation, use an object library instead: add_library(${name} OBJECT ...).
You can also remove the boilerplate for the main function from the test files.
WILL_FAIL can easily turn your tests into watermelons (green on the outside, red on the inside): If compilation starts to “fail” for an unrelated reason (like “failing to add the library dependency”), the test will signal success even when the code under test breaks its promise. This is worse than having no test at all, because it gives you a false sense of confidence during refactoring. It is better to set PASS_REGULAR_EXPRESSION to make sure the compilation fails for the right reason. This may be simplified with C++26, where you use = delete("your own message").
Right – this was indeed happening to me and had me confused for awhile until I ran ctest –verbose and saw the compile was failing because the include path was wrong. I needed the safe_bool dependency to bring in the path.
The trouble I have with the regular expression – well you know it: first you have a problem, then you decide to solve it with regex, now you have 2 problems But seriously, the cmake docs are minimal to start and seriously bad here. What regex language is used? Not a single example of a actual regex – just a list of strings – which are, I guess ‘contains matched against output’? It’s not said. Maybe it’s somewhere and I missed it.
Still, you’ve convinced me it’s better – I’ve had to get over a lot of under documented cmake in my life already, this will just be one more…
Indeed – I plan to show that off in my lightning talk next week. Since CI is currently complaining about failing on older language standards it got me thinking what I should target. It’s simple enough that different implementations for different cpp versions would be fine - that’s probably where I’ll go.
I just had the problem that adding a subproject with add_subdirectory(EXCLUDE_FROM_ALL) will cause ctest to fail after make all because the test targets are not built. I worked around the issue by building the test targets with test fixtures in the subproject:
The surprising overlap is that for this use case too, I need to define a test for building a target, just like for the usecase of compile tests and compile fail tests. The test command is 100% identical.
But seriously, the cmake docs are minimal to start and seriously bad here. What regex language is used? Not a single example of a actual regex – just a list of strings – which are, I guess ‘contains matched against output’? It’s not said. Maybe it’s somewhere and I missed it.