What do we want our presets to be?

Beman Project Development
1

@river landed CMake preset support in beman.exemplar and one can now build and test with cmake --workflow --preset gcc-debug. This works great on Linux, but that’s about the only platform it supports right now.

The intent with these presets is that they are a way for developers to get up-and-running with a best-practice project quickly. The ideas is to eliminate a confusing array of choices. The gcc-debug preset, for example, has several sanitizers turned on . . . which is plainly a good practice for development.

The question remains, however, what should our list of presets be? Here are some ponderings about that.

One answer would be “we should have the most popular generator with the most popular compiler on each of the three major platforms”. That would lead to something like this:

Platform Presets provided Explanation
Linux gcc-debug Ninja MultiConfig, sanitizers, etc.
gcc-release RelWithDebugInfo, -O3, etc.
MacOS xcode-debug XCode, Apple clang w/ sanitizers, etc.
xcode-release RelWithDebugInfo, -O3, etc.
Windows vs2022-debug VS2022, TBD
vs2022-release TBD

Why include release variants? When developing C++ libraries it is frequently important to measure performance and inspect the generated assembly of optimized code.

There are other ways we could cut this. We could, for example, provide presets for Ninja on every platform. That would be great for the Linux-centric folks, but will be a turn-off for people who primarily develop using the popular IDEs on those other platforms.

What about someone who wants to use clang on Windows or any other number of dizzying variations. The message would be that those folks can call CMake directly however they like or supply their own CMakeUserPresets.json file. The goal here is to avoid surfacing complexity and capturing the 90% use case.

1 Like
2

Do you know Organizing CMake presets? (without workflow presets)

and cmake-init, a CMake project initializer (without cxxmodule support yet)

You should not forget the clang++-18 or newer compiler which is available on all OS in your list.

clang-19 has support for c++23 std modules!

3

clang-19 has support for c++23 std modules!

see too C++20 Modules, CMake, And Shared Libraries

I would use more general presets and in combination with toolchains and environment variables, all is possible without combinatorial explosion!

User may use CMakeUserPresets.json like cmake-init recommends.

I have some examples running. :wink:

4

I would love to contribute by implementing more preset, given the complex state of complier support, I wonder if we should first implement tool chain files with CMAKE_CXX_FLAGS_Debug_INIT as suggested by @bretbrownjr in the original create CMake preset PR.

The suggestion were:

You can use variables like CMAKE_CXX_FLAGS_Debug_INIT to tune what a Debug or Release build entails, for what it’s worth. It’s maybe a little nicer to use those variables instead of CMAKE_CXX_FLAGS. But not a huge deal.

We are already seeing sanitizers getting disabled across board in preset due to Apple Clang not supporting leak sanitizer. Plus, we may need to conditionally pass in different flags when library develops need cutting-edge compilers. Looks like right now is the best time to implement tool chain files for standard Debug/ Release config to avoid combination explosion (e.g. macos-gcc-debug).

5

I am struggling to see how module support is relevant to CMake presets, can you provide more explanation here?

7

Not through a tool chain file, but selective sanitizer enabling has been implemented at: Specify sanitizer parameters in CMake by wusatosi · Pull Request #76 · beman-project/exemplar · GitHub .

I can come and implement more presets once this gets merged. We have the infrastructure to introduce more presets now.

8

I would suggest that sanitizer builds should be part of the normal dev workflow, not something that is CI only. Fixing an msan, asan, ubsan, tsan, rtsan …, bug is easiest right after you introduce it. Using asan and ubsan is my personal default for running tests. I don’t have any threads, and msan doesn’t compose well, although I should probably look again.

Note: this is a request to make sure what ever mechanisms we use in CI can be easily replicated on my laptop, not a requirement for anyone to change their personal software process. Or PSP if you’re fancy.

9

On exemplar, gcc-debug runs the project with address, undefined, and leak sanitizer as default.

Specify sanitizer parameters in CMake by wusatosi · Pull Request #76 · beman-project/exemplar · GitHub is literally to refactor exemplar so that CI and preset use the same sanitizer setting script. Go check it out and give me feedback :slight_smile:

10

There should be PRs ready that adds:

  • xcode-debug
  • xcode-release
  • msvc-debug
  • msvc-release

along with appropriate CI testing.

I will add a follow-up PR that: fails with a message if you are using gcc-* preset but the underlying compiler is apple clang (apple clang trying to disguise as gcc). I think this would be a significant enough use case to warren this check, but I don’t think we are detecting for every erroneous use.

I will post the PRs to exemplar tomorrow.

It’s out! Go check it out: Adds xcode + msvc preset by wusatosi · Pull Request #82 · bemanproject/exemplar · GitHub

Oh btw, should we add clang-debug/release, this should be trivial?

11

Yes, I always use on each OS the newest clang++ compiler, clang-tidy, and clang-format.

On Windows MS Visual Studio 2022 comes with clang-17 and I use ninja with msvc and clang-tidy.

On OSX I have clang-19 installed with brew.

12

I have tested if it is possible to have always the same N presets on all build OS hosts.
Yes, it is!
sse Feature/export cmake config package by ClausKlein · Pull Request #14 · bemanproject/inplace_vector · GitHub

13

You should also note this Hide workflow presets with configure stages disabled by a condition when using --workflow --list-presets

14

I have created a prove of concept preset set at execution26.

I created the simples preset possible: a generic one with only tow workflows debug and release which works on each OS and with each toolchain. Just as demo and to have one.

The preset used in examplar will be more and more complex and unreadable!
It needs a workflow for each compiler and each build config type that should be supported.

At inplace_vector I developed an other one with more generic presets:

bash-5.2$ cmake --list-presets
Available configure presets:

  "debug"   - Debug Build
  "release" - Release Build
  "gcov"    - Coverage Build
  "asan"    - AddressSanitizer Build
  "lsan"    - LeakSanitizer Build
  "usan"    - UndefinedSanitizer Build
bash-5.2$
15

First library with FILE_SETS CXX_MODULES:

see First step to install cxx_module by ClausKlein · Pull Request #92 · bemanproject/execution26 · GitHub
:sunglasses:

16

Note this one-click setup has been broken by Inject FetchContent logic by bretbrownjr · Pull Request #105 · bemanproject/exemplar · GitHub knowingly.

And current preset design would likely not be able to continue due to combination explosion brought by the packaging system.

17

We could simplify things a lot if we picked one or two cross platform package mangers to support for default workflows. Among other reasons, the workflows would be consistent across all platforms, providing simplicity for Beman maintainers, Beman users, and the tooling that supports all of the above.

So I’d rather eject some other use cases than drop support for packaging workflows from preset support.

I don’t think we need to feel especially responsible for the tension here. It’s an issue in upstream CMake itself (and pretty much every other build system besides). We should just do our best to keep it easy and provide docs for extra workflows to the extent that we can’t justify more complexity in the preset JSON.

1 Like
18

My personal favourite would be simply like this:

{
  "version": 9,
  "cmakeMinimumRequired": {
    "major": 3,
    "minor": 30,
    "patch": 0
  },
  "include": [
    "cmake/CMake${hostSystemName}Presets.json"
  ],
  "buildPresets": [
    {
      "name": "debug",
      "configurePreset": "debug",
      "configuration": "Debug",
      "targets": [
        "install"
      ]
    },
    {
      "name": "release",
      "configurePreset": "release",
      "configuration": "Release",
      "targets": [
        "all_verify_interface_header_sets",
        "install"
      ]
    }
  ],
  "testPresets": [
    {
      "name": "test_base",
      "hidden": true,
      "output": {
        "outputOnFailure": true
      },
      "execution": {
        "noTestsAction": "error",
        "stopOnFailure": false
      }
    },
    {
      "name": "debug",
      "inherits": "test_base",
      "configuration": "Debug",
      "configurePreset": "debug"
    },
    {
      "name": "release",
      "inherits": "test_base",
      "configuration": "Release",
      "configurePreset": "release"
    }
  ],
  "packagePresets": [
    {
      "name": "release",
      "configurePreset": "release",
      "configurations": [
        "Release"
      ],
      "generators": [
        "TGZ"
      ]
    }
  ],
  "workflowPresets": [
    {
      "name": "debug",
      "steps": [
        {
          "type": "configure",
          "name": "debug"
        },
        {
          "type": "build",
          "name": "debug"
        },
        {
          "type": "test",
          "name": "debug"
        }
      ]
    },
    {
      "name": "release",
      "steps": [
        {
          "type": "configure",
          "name": "release"
        },
        {
          "type": "build",
          "name": "release"
        },
        {
          "type": "test",
          "name": "release"
        },
        {
          "type": "package",
          "name": "release"
        }
      ]
    }
  ]
}
1 Like
19

Wow, I had no idea ${hostSystemName} was something that could be used. That’s so much better than what we’re currently doing. Thanks @ClausKlein!

20

the files behind are found execution

But I used a different way to select the compiler:
either you set CXX i.e.: export CXX=g++-15
or let cmake use the system default.

Only for windows, cl is set!
I would recommend to use GitHub - MarkSchofield/WindowsToolchain: A repository containing a CMake toolchain for using MSVC for Windows.

It would also be possible to use toolchain files.

But we need to use cmake v3.30 !

21

A good entry how Organizing CMake presets

But also note Hide workflow presets with configure stages disabled by a condition when using `--workflow --list-presets` (#26236) · Issues · CMake / CMake · GitLab