Introducing a C++ LaunchPad for the Beman Project

Abstract

The C++ LaunchPad is proposed as a foundational component for the Beman Project to support the rapid adoption and evolution of C++ standards and tooling. By providing a dynamic, easily upgradable framework, the LaunchPad addresses the critical need for developers to stay aligned with evolving best practices, tools, and workflows over time. This proposal highlights why such a solution is essential for the long-term success of the Beman Project’s objectives and the broader C++ community.

Problem Statement

As the Beman Project evolves, it aims to support the incubation of libraries that adhere to evolving C++ standards. However, the long lifecycle of C++ libraries and tools presents several challenges:

  1. Tooling and Automation:

    • Tools like linters, formatters, sanitizers, and CI/CD pipelines improve over time. However, integrating these advancements into older projects is often non-trivial.
    • Many developers lack a clear pathway to adopt updated tools, leading to technical debt.
    • Forking a classic “one-off” template project for Beman libraries can assist with initializing a new library, but it significantly increases the maintenance burden, which hinders future improvements.
  2. Consistency Across Lifecycles:

    • The Beman Project supports libraries across at least two C++ standards (a 6+ year span). Without a reliable way to align libraries with evolving practices, inconsistencies and inefficiencies can emerge. The pipelines, tools, and best practices that each library follows are often tied to the state of the template project at the time of the library’s creation.
  3. Onboarding and Developer Experience:

    • A smooth starting point and ongoing support are critical for encouraging adoption by the broader community.
    • While they are still free to configure some of the default options that are provided by the LaunchPad, developers need a centralized, opinionated solution to streamline their workflows and ensure compliance with modern standards.
  4. Evolving Best Practices:

    • New C++ standards introduce idioms, guidelines, and features that redefine best practices (e.g., C++20’s modules, coroutines, ranges).
    • Developers need an easy way to integrate these changes retroactively into existing projects.

Proposed Solution

The C++ LaunchPad is a framework comprising two main repositories: the core infrastructure repository and the template project (exemplar) repository. The template repository is designed to set up new libraries and incorporate the core repository in a modular, upgradeable way. For the Beman Project, a library template will be provided, with the potential for additional templates for other use cases (e.g., CLI, GUI, service) to be added in the future. This framework is intended not only as a starting point for new projects but also as an upgradable foundation for existing ones.

The LaunchPad will:

  1. Provide an Upgratable Core:

    • Encapsulate best practices, modern C++ idioms, and reusable components.
    • Allow retroactive updates by enabling developers to pull in new features, tools, and improvements without disrupting domain-specific code.
    • Provide an automated way to upgrade a library while being non-intrusive to the development lifecycle of the developer. This mechanism could take effect after a successful release. Additionally, developers should have the option to perform a manual, on-demand upgrade.
  2. Centralize Tooling and Automation:

    • Include CI/CD pipelines pre-configured with linters, sanitizers, formatters, and testing frameworks.
    • Integrate tools like CMake, Conan, CPM, and FetchContent to ensure compatibility and ease of use.
  3. Simplify Adoption of C++ Modules:

    • Serve as a practical example and implementation of C++20 modules, showcasing how they can be effectively used in production.
  4. Encourage Standardized Workflows:

    • Promote consistency across projects incubated within the Beman Project by enforcing standardized structures and tooling configurations.
  5. Support Long-Term Maintenance:

    • Provide a mechanism for the Beman Project to push updates (e.g., new linting rules, pipeline tasks, or dependency versions) to downstream users.

Key Features

  1. Dynamic Framework:

    • Utilize CMake with tools like CPM or FetchContent for modular, version-controlled dependencies.
    • Enable users to pull updates from the LaunchPad repository while preserving domain-specific customizations.
  2. Preconfigured Pipelines:

    • Example CI/CD setups for GitHub Actions, GitLab CI, and other popular systems.
    • Built-in support for modern C++ workflows, including sanitizers, code coverage, and benchmarking tools.
  3. Developer-Centric Automation:

    • Automated linting, formatting, and testing to improve code quality.
    • Pre-integrated tools for modular builds using C++20 modules.
  4. Scalability and Extensibility:

    • Designed to scale with projects of any size and adapt to future tooling advancements.
  5. Environment Consistency:

    • Solve “Works on my machine” problems during development by providing dev containers ready to be used for any OS.
    • Support for any major standard-conformant compiler and OS with CI/CD.
    • Works with any IDE that opens CMake projects.

Benefits

The idea of a LaunchPad repository for Beman libraries with features for both initial project setup and retroactive updates has several potential benefits:

  1. Streamlined Evolution:

    • Libraries and tools incubated by the Beman Project can evolve in tandem with C++ standards without significant effort from developers.
  2. Increased Productivity:

    • Developers spend less time configuring and maintaining their projects, focusing instead on domain-specific logic.
  3. Improved Adoption of Modern C++:

    • By providing a practical example of best practices and tools, the LaunchPad encourages broader adoption of modern C++ features like modules.
  4. Enhanced Developer Experience:

    • Simplified workflows and standardized processes make it easier for contributors to onboard and collaborate.
    • Simplified Onboarding: New contributors can quickly get started with a well-defined project structure and pre-configured tooling.
    • Reduced Boilerplate: Automates the creation of common files and configurations, saving developers time and effort.
  5. Standardization:

    • Consistent Project Structure: Ensures all Beman libraries adhere to a common project structure, making it easier to navigate, understand, and contribute to them.
    • Uniform Tooling: Provides a consistent set of tools and build systems across all libraries, simplifying development and maintenance.
  6. Enhanced Maintainability:

    • Retroactive Updates: Allows developers to easily update existing libraries with the latest best practices, security patches, and tooling improvements.
    • Centralized Management: Provides a central location for managing and updating project templates, ensuring consistency across all libraries.

Further Exploration

Steps for Implementation

  1. Research Existing Tools and Platforms:
    • Investigate similar solutions successfully adopted by other open-source projects. This will help identify best practices, potential pitfalls, and tools that can be adapted for the LaunchPad.
  2. Repository Setup:
    • Create a public GitHub or GitLab repository for the LaunchPad.
    • Include initial configurations for C++20 modules, CI/CD pipelines, and common tools.
  3. Community Involvement:
    • Gather feedback from Beman Project contributors and the broader C++ community.
    • Iterate on the design and features based on real-world use cases.
    • Engage library developers to understand their specific needs and challenges. Their insights will ensure the LaunchPad addresses real-world problems and gains community support.
  4. Versioning and Updates:
    • Establish a versioning system to manage updates to the LaunchPad.
    • Provide detailed documentation and migration guides for users.
  5. Develop a Prototype:
    • Create an initial version of the LaunchPad repository to demonstrate its functionality. Use this version to gather user feedback and refine it based on their suggestions.
  6. Promotion and Integration:
    • Promote the LaunchPad as the recommended starting point for new Beman Project libraries.
    • Integrate its use into official Beman Project guidelines.

By incorporating these steps and fostering active participation from the Beman community, the LaunchPad can become an indispensable resource that greatly enhances the project and empowers its developers.

Implementation Considerations

  • Automation: The repository should be designed to automate as much of the project setup and update process as possible, using tools like cookiecutter or Yeoman to generate project templates and submodules, FetchContent, custom tools to upgrade them.

  • Flexibility: The repository should be flexible enough to accommodate the diverse needs of different Beman libraries while still maintaining a degree of standardization.

  • Structure: The folder structure serves as an interface between the framework and the library. While the community has established a “common” folder structure for libraries, there is also a “canonical” form (canonical structure proposal) with advantages and disadvantages that need to be carefully considered.

  • Vendor: The framework should avoid relying on technologies tied to a specific vendor unless that vendor is widely recognized as the community standard. This principle ensures the LaunchPad supports a broad range of compilers, IDEs, package managers, and code-hosting platforms. However, given limited resources, development can initially leverage the maintainers’ existing expertise, provided it does not hinder the future inclusion of alternative vendors in different areas.

  • Workflow: The framework should align with the library’s workflow. I assume that a small core team (2–5 people) will exist for each library, and a simple feature-based workflow will be used. Libraries will also be open to “external” contributions, meaning the fork workflow should be supported.

  • Cost: While some resources are free, pipelines generally incur costs. To optimize expenses, ensure all jobs can run locally, maintain a small core team, and enforce the requirement for a full pipeline to pass before approving a pull request.

  • Documentation: Clear and comprehensive documentation is essential to guide developers on how to use the repository and effectively manage their projects.

  • Interface: Apart from the folder structure, there are “things” that must be controlled solely by the maintainers, “things” that must be set to reasonable defaults by the maintainers, and “things” that developers should be able to override consistently. The contract specifying who defines what should be clarified in a well-structured manner.

  • Community Involvement: The Beman community should be actively involved in the design, development, and maintenance of the repository to ensure it meets the needs of library developers.

Conclusion

A LaunchPad framework with the features described above would be an invaluable asset to the Beman Project. By offering a standardized and streamlined development experience, it can enhance the quality, consistency, and maintainability of Beman libraries.

The C++ LaunchPad serves as a crucial tool for ensuring the long-term success of the Beman Project. By providing a dynamic, updatable foundation, it directly addresses the challenges of evolving best practices, tools, and workflows. This framework empowers developers to stay aligned with modern C++ standards while maintaining a cohesive and productive workflow. It enables them to really focus on what matters the most. The LaunchPad not only accelerates the development process but also solidifies the Beman Project’s position as a leader in advancing the future of C++.


Be the plus to the C++!

With regards,

Jason

This appears to be a lot of AI generated text and I don’t understand the purpose of this post. Could you please clarify your intent @Jason5480?

Hello @dsankel, thank you for looking into that and sorry for obfuscating the main idea through AI, I had the impression that this would make it better, but anyways… What I want to address with this post is that having a simple starting template project for the Beman project libraries is not enough!
Let’s assume I want to create a new library today and I use the exemplar as a starting template. After a couple of months this template evolved, some new tools and pipelines were introduced, some defaults changed and I want to also include those changes in my library right after I finish the development of the ongoing release.
So, at this point I have to track, check and apply all the changes that happened in the exemplar repo during the period that I developed my library. If this is done manually for one, or maybe a couple of repos it’s fine, but in general it doesn’t scale in a multi-repo approach.

What I propose here is to have a (semi-)automatic mechanism that updates at the end of a development lifecycle the library projects started from the exemplar template. That way the “new staff” from the template will be included in the existing libraries that are still in development, raising the quality standards as time passes and keeping the Beman libraries consistent.
The update should be as non-intrusive as possible and this is why it should be done after a successful release. At this particular moment the library developer has the time to inspect what new tools and pipelines were introduced, fix warnings and investigate potential improvements.
For example, if a new tool like klocwork was introduced in the pipelines (that checks some Misra rules), at the middle of the development, then most probably new warnings will arise, forbidding the release pipeline to run successfully and forcing the developer to solve all the issues before. This will cause frustration to the library developer and he will most probably opt-out and we don’t want that. On the other hand, we don’t want the library developer to only update his library manually, since it is easy to “forget”.

As discussed in this topic Should we have a dedicated repo for infrastructure? - #3 by bretbrownjr, I think that having a core infrastructure repo will decouple the DevOps “staff” needed for the maintenance of all library repos, from the development “staff” needed for a specific library. The infrastructure repo can provide the core DevOps functionality of each library and will get updated at regular intervals as described above, were the exemplar repo will be used as a starting template, but with a twist, the infra repo will be used as a “dependency”.

I don’t want to get into more technical details like if this will be used as a git submodule, downloaded using CMake FetchContent, or as a package, or what are the exact DevOps functionalities that should be provided by the infra repo, or what library specific infrastructure configuration should be, since each one of them might be a different topic by itself.

My vision is to enable developers to use the latest quality standards and tooling without much effort. I hope I made it clear this time what my intention is and what I want to achieve.

Thank you for keeping track with the idea,

Hi @Jason5480! It sounds like quite a few of us are on the same page about where we want to go with the development and user experience for Beman projects. I’m glad to have you on board, in the discussions, and contributing changes.

It seems clear to me that keeping the amount of detail in each projects small and focused on describing each project individually is where we want to be. If we have extensive procedural logic to support different style choices, workflows. and basically anything else that isn’t C++ code, we’ll have to churn all of the projects until we find a sweet spot – if we find a sweet spot.

Anyway, feel free to propose specific design changes to exemplar. It could look like a PR to exemplar that suggests a bigger change. It’s also fair to come here to the Beman Discourse describe a change you’re thinking about making, especially to save you the trouble of implementing something that nobody is really excited about in the first place.

And to drill in on that last point, keep in mind that bigger design changes come with some risk – that they don’t meet expectations or that they’re just too much for the current community to get behind. It’s not in the core goals of the Beman Project to innovate in tooling and project style as such, though it seems clear to me that some amount of trailblazing will be warranted because there isn’t a boring enough set of portable best practices yet.

But, bottom-line, I think there are quite a few of us with similar feelings about needing to help the C++ ecosystem iterate on tooling and project style. I’m interested to see what you’d like to do specifically. Let us know what you need to keep you productive, encouraged, and included.

1 Like