A good case study I haven’t seen mentioned: NVIDIA has a suite of libraries that they release under their “rapids” org, which resolve both 3rd party dependencies and dependencies on each other via CPM. The common cmake code to support that lives under “rapids-cmake”:
https://docs.rapids.ai/api/rapids-cmake/nightly/cpm/
It’s a good example of what CPM can look like when done right, and with a scale and scope that is similar to what Beman will face longterm.
I was helping slightly on a stdexec PR lately. The rapids framework didn’t handle FILE_SET HEADERS properly for some reason. It wasn’t clear how I could assist with that issue without delving into the rapids framework in ways I didn’t have time for.
I’m not opposed to supporting CPM, but I do not want mentions of rapids, CPM, or any other divergent “style” in CMakeLists.txt. I want libraries to be portable to any given dependency management system without having to add a variety of hooks specifically for each supported ecosystem.
One of my criticisms of Conan 1.0 was that it tended to affect the style of CMakeLists.txt, a drawback that Conan 2.0 seems to have addressed.
Good point.
Welcome aws-nslick! I’m not an expert in cpm, but it looks to me like for the project you cite, the dependencies are externalized into json and then generated into cmake – or something like that? Because I think normally @bretbrownjr is correct that you’d add CPMAddPackage directly into cmake to organize the fetch content.
Re: changes that span multiple repositories, I have had similar experiences to @bretbrownjr. Is there any interest in creating a beman.superbuild project that builds everything in beman using a single top-level project with the individual beman projects as sub-projects? In the past I’ve used mr to implement a solution like this. The workflow is pretty straightforward.
% git clone <super-build-url>
...
% mr clone
....
% cmake -S . -B build
...
% cmake --build build
My experience has been that when the projects all follow the same conventions, it’s straightforward to make building specific targets, executing specific tests, etc intuitive. For example, when building a super-build:
% cmake --build build --target tests # runs all tests in all projects
% cmake --build build --target <subproject>.tests # runs only the tests for <subproject>
The nice thing about this is that the individual project builds can define the same set of test targets (in other words, provide the same build interface), with the only difference being that in the case of an individual project build the two commands above are semantically equivalent (i.e. they both run the tests for only that project).
Another nice thing about this approach is that we can use project-scope variables in such a way that, when not defined, they inherit their values from super-build scope variables. For example, if we have a subproject <X>, the variable BEMAN_<X>_BUILD_TESTS, if not explicitly defined, would inherit the value of BEMAN_BUILD_TESTS. This allows us to easily configure all projects in the super-build to build tests like so.
cmake -S . -B build -DBEMAN_BUILD_TESTS=ON
And it preserves granular control of individual projects. For example, we can build tests for all projects except Y using the following command.
cmake -S . -B build -DBEMAN_BUILD_TESTS=ON -DBEMAN_<Y>_BUILD_TESTS=OFF`
This is the approach I had in mind when I submitted this PR to introduce config-file package creation to the exemplar.
This is sorta what Boost is doing, but with b2 and not cmake. They have cmake support, but it’s kinda limited.
The link is to rm, but I assume you mean https://linux.die.net/man/1/mr
I think this is interesting and is maybe orthogonal to the discussion of packaging via vcpkg, conan, etc and dealing with getting the dependencies. It’s also not going to be a cross-platform option I don’t think.
I’ll comment on the PR.
It’s possible to implement what @chayden83 outlines using a standalone project with any of:
FetchContent callsCMakeLists.txtOf course it’s possible to do more than one of those if it’s interesting.
All of the above should “just work”, including for working on specific subsets of dependencies, especially if there is CI set up on that superbuild project. The options leveraging package managers will have the most powerful options, of course. The other ones are bespoke implementations of our own dependency management systems in one way or another.
Note that the CMakeLists.txt in beman and the libraries.json in beman combine to create a FetchContent oriented superbuild. I initially whipped that up as a proof-of-concept (and maybe a starting point) that could support multiple dependency management implementations, but it hasn’t gotten any momentum from there. But maybe we’re ready to revisit this approach.
One missing element from the Beman Standard that makes these approaches maintainable is a requirement that we start using GitHub releases to actually version different revisions of our projects. There are other benefits to doing actual releases beyond superbuild workflows, so I highly recommend this as a next step for the community. See this issue on exemplar.