Subproject structure:
The Beman template should include:
/include(/beman?)
/src (if applicable)
/examples
/test
/doc
/paper
CMakeLists.txt
I’d advocate for
/include // public interface
/build // source code if applicable
CMakeLists.txt
/doc
/xml
/html
/pdf
/paper
// other subdirectories each with their own CMakeLists.txt
/test
/example
/profile
/benchmark
...
I moved your post to the top level since this is a sizable topic in itself.
I like these ideas.
@ramey, why does your “should include” have /paper and the “advocate for” have doc/paper?
It seems like some Beman libraries will include the source for the paper they implement, but others will not. I like the idea of having a link to the official paper through wg21.link in the top level README.md as recommended by [README.IMPLEMENTS]. If 99% of users access the paper through that link, I think something like your second suggestion (putting the paper in doc/paper) makes the most sense.
I like the idea of putting examples/ in the top level (as opposed to src/examples) since it will be immediately obvious for someone looking at the github repository where they can find some example code.
This part of directory structure is inconsistent across boost libraries. This hasn’t created any major problems so there hasn’t been any real motivation to try and address that.
This is what I use in the two libraries I maintain. It’s distinguished from other libraries in that the both the original xml and the generated html and pdf are part of the package. I’ve always felt that a user should be able to clone or acquire the zip of the library and immediately begin to peruse the documentation on the package he downloaded. That is, everything about library should be available in one spot.
The usage of Boost Book - a masterpiece by Doug Gregor - clearly envisioned using xslt to create a boost master consolidated document while still maintaining everything contained in each library package. Quickbook by Eric Niebler addressed the issue of generating XML in human friendly way. The usage of XML which maintains content independently of presentation is/was a very good idea. It’s the reason that my libraries include PDF, html, and if desired ebook formatted documentation with “zero effort” using xslt. (xslt is another beast) Unfortunately the world has moved on from xml (presumably due to the difficulty of editing its torturous syntax) and left aside the fundamental idea of maintaining content substance orthogonally to content formatting. So now we have a variety of convenient mark down language and hacky tools like Doxygen and halfassed literate programming by including doc/formating info in C++ comments.
Robert Ramey
Yep, that is the state of things AFAICT.
Sean Parent and I investigated Rust’s documentation situation. They standardized on a doxygen-like, in-source markdown for API reference docs (see rustdoc). For more extensive tutorials and user guides, they use a separate mdBook tool which also uses markdown. Both have automated testing of code snippets.
Clap is a good example. Their API reference docs include simple tutorials and cookbooks, but there’s a separate Command line apps in Rust book with in-depth topics and recommendations.
Inline documentation is great for IDE tooling and is easy to keep up-to-date with the code. On the other hand, when it gets too long it can get in the way of code editing. I wonder if something like Rust’s “inline docs” + “external guides” is the best way to go.
@sean-parent, do you have anything to add?
I should also @InbalL since she volunteered to look into Doxygen at today’s weekly sync meeting.
Just another option to add. Parts of Adobe use a tool, hyde that uses the clang libTooling to generate YAML (it can also generate JSON) front matter that can then be consumed by jekyll. There is a jekyll theme that can generate the docs with a lot of flexibility. I wish I had a good example, but I recently started a significant project to update the stlab.cc docs to hyde 2.0, and there isn’t a lot to show yet (there is a lot more use internally by I can’t link to those examples). The tool is powerful and flexible, can validate and update docs in place, and has limited support for doxygen-style comments if you like literate programming. Currently, it only supports C++17 constructs (it doesn’t error on C++20 constructs like concepts; it just ignores them) and has a few other minor gaps (it doesn’t handle inline namespaces well or variables). But the code is pretty simple to update for these things. We just haven’t gotten to them. My goal is to match the quality of structure you find on cppreference, plus add tutorial templates. Hyde was developed after spending years trying to get Doxygen to do what we wanted and failing. I’m happy with it, but it would require some PRs before considering it fully baked.
Just a note on this. With most libraries these days the convention with cmake is to make a build directory, change to that directory, and do cmake ... Then all the build products are under the build directory and can be deleted trivially. That’s why the template has src instead of build for source.
I’d also recommend looking at pitchfork: GitHub - vector-of-bool/pitchfork: Pitchfork is a Set of C++ Project Conventions – I think the proposal above is following the spirit of the pitchfork recommendations which are built to enable tooling.
I prefer examples in The place I suggested because of my personal experience there are a number of different tests that I wanna run. I want to run, of course the normal suite. But also I want to run all the examples that I’ve used in the documentation. I also want to run a number of test that I don’t run very often which are performance test. The reason they don’t run very often because they take disproportion amount of time and once I’ve responded to profile the first time it doesn’t really change much unless there’s a larger change in the library. Finally, it leaves the door open to anybody else who has some special tests that he wants to separate from the test suite. This could be for example, special test to work with MPI or something like that. Finally, I liked the symmetry of having alternative test suites be at the same level as the fundamental official suite. Simplifies the script and permits easier expandability to other tests at zero cost. For what it’s worth that’s my rationale.
My complaint with Doxygen is actually pretty straight forward: I could never find a straightforward way to define type requirements. This meant I could never really document a template. Yeah, I’m aware that there’s probably template keyboard within oxygen. But still, last time I looked at it, it was very much of the model that types our specified in terms of their variables and dependent functionsway. And this made it difficult to specify type parameters, which are really defined by their behavior rather than their function and data members.
Cmake permits, one to specify an out of source directory in the command line for building, etc. His permits one to build and test even if one only has read only access to the library source directory. Cmake has the idea that subdirectories contain their own CMakeLists.txt file. I actually like this because it permits, moving things around without really editing and having things still continue to work. I did create a separate directory called for including the CMake filed. so far so good. But I couldn’t include within the project subdirectories different CMakeLists.txt filed. All the information that would’ve been in separate CMake directories had to be included in the “top level“ cmake/CMakeLists.txt directory. Kind of sloppy in my opinion.
Understood – I’m just saying that ‘build’ shouldn’t be the name of the directory for the .cpp files because it’s frequently used to in fact build this part of the subtree.
I’m not sure what you’re getting at here. In a large mutli-library, multi-exectuable type cmake setup there’s usually details at the library/executable level and yes a top level CMakeLists.txt that drives a comprehensive build if that’s what you want.
I found this project tree, but I’m wondering why src and tests are such deep trees without any real need.
bash-5.2$ tree -d -L 4
.
|-- bin
|-- cmake
|-- docs
|-- etc
|-- examples
|-- include
| `-- beman
| `-- execution26
| `-- detail
|-- src
| `-- beman
| `-- execution26
`-- tests
`-- beman
`-- execution26
`-- include
17 directories
bash-5.2$ git ls-files '::*.in' '::*.cmake' '::*CMake*'
CMakeLists.txt
CMakePresets.json
cmake/CMakeDarwinPresets.json
cmake/CMakeGenericPresets.json
cmake/CMakeLinuxPresets.json
cmake/CMakeWindowsPresets.json
cmake/Config.cmake.in
etc/ci-clang-toolchain.cmake
etc/clang-16-toolchain.cmake
etc/clang-17-toolchain.cmake
etc/clang-18-toolchain.cmake
etc/clang-19-toolchain.cmake
etc/clang-flags.cmake
etc/clang-toolchain.cmake
etc/gcc-11-toolchain.cmake
etc/gcc-12-toolchain.cmake
etc/gcc-13-toolchain.cmake
etc/gcc-14-toolchain.cmake
etc/gcc-flags.cmake
etc/gcc-toolchain.cmake
etc/llvm-16-toolchain.cmake
etc/llvm-master-toolchain.cmake
etc/llvm-toolchain.cmake
etc/toolchain.cmake
examples/CMakeLists.txt
src/beman/execution26/CMakeLists.txt
tests/beman/execution26/CMakeLists.txt
bash-5.2$
I think we decided this just for 1) consistency and 2) some IDEs work better if public interface code and implementation code files have same structure. @Sdowney , do we have other reasons?
It’s at least partly because all the original code did not separate source into all these different directories. There was just src. Then we got include and then tests. This will get even more complicated when we get module interface and implementation units.
On my todo list is to update the standard to allow single tree rather than split sources, as in the merged form of Pitchfork
Keeping names from colliding across projects is the underlying reason.
It helps when doing aggregate work, when you want the dev feedback loop even tighter when making changes across components. For that sort of work pulling in repositories via git-subtree and having a single tree, rather than vendoring in via git-submodule, is a useful workflow.
It also makes checking that namespace is correct much more mechanical.
Which IDE is so stupid?
Even GVIM mit ctags is able to navigate and build my sources!
find related file, vs find definition. It’s more about where something ought to go, and agreement on that, really, than layout, and making review easier.
If src/thing.cpp doesn’t live in namespace beman::thing is that an accident, is it on purpose? On the other hand, if I’m looking in a code review for my/project/meow.h and want to see docs for beman::optional26::optional it’s a lot easier if I know where that’s going to live.
Problems of scale. I have 500K files to deal with. Even with good source indexing tools, organization by convention makes life easier, and prevents all sorts of accidental collision. Working the same way on small scale makes it easier to move things in and out of the large scale I live in for work.
Also, see Verify headers by steve-downey · Pull Request #75 · beman-project/optional26 · GitHub which gets rid of the empty optional.cpp. Although if there were anything that should be in the cpp file, having it’s name match the header would put it in the same place.
It does end up putting a CML.txt in the include directory that will not be part of the deployed set. include isn’t part of the name of deployed file, though, it’s part of INCLUDEDIR, which defaults to PREFIX/include, but is (usually) deployer configurable.
Hmm, I wonder if we really want that? If we do I guess I’d like to see it consistently, but what is a bit different there is that the user headers aren’t in the /include directory.
w.r.t the rest of the tree, my take is that the include sub-tree is the only place we need to have the structure. Especially for stuff that’s header only /tests and /example are typically flat (or broken up by some domain specific factoring) in Boost.