# Using shell scripts in .github/workflow

**URL:** <https://discourse.bemanproject.org/t/using-shell-scripts-in-github-workflow/384>\
**Category:** Beman Project Development\
**Created:** [March 22, 2025, 4:47pm UTC](https://discourse.bemanproject.org/t/using-shell-scripts-in-github-workflow/384 "2025-03-22T16:47:08Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![Sdowney](https://yyz1.discourse-cdn.com/flex029/user_avatar/discourse.bemanproject.org/sdowney/32/73_2.png) [@Sdowney](https://discourse.bemanproject.org/u/Sdowney)\
**Post date:** [March 22, 2025, 4:47pm UTC](https://discourse.bemanproject.org/t/using-shell-scripts-in-github-workflow/384/1 "2025-03-22T16:47:08Z")

</div>

I’ve seen a few projects in the wild putting the code for part of a CI action in a shell script, rather than putting it inline in the yaml for the action. On the `pro` side that makes it much more testable outside a github action, on the con side it’s a layer of indirection and one more thing to support.

Does anyone have strong feelings about not doing it, before I dive in and pull out some of the `optional ` run blocks into a separate shell script?

For example: [draft/.github/workflows/check.yml at main · cplusplus/draft · GitHub](https://github.com/cplusplus/draft/blob/main/.github/workflows/check.yml#L34)

> ```
> - name: check-output.sh
> run: ../tools/check-output.sh
> 
> ```

rather than putting all of [draft/tools/check-source.sh at main · cplusplus/draft · GitHub](https://github.com/cplusplus/draft/blob/main/tools/check-source.sh) inline which clearly would not scale.

or

> <https://github.com/catchorg/Catch2/blob/devel/.github/workflows/validate-header-guards.yml#L32>

> ```
> - name: Check that there are no duplicated filenames
> run: |
> ./tools/scripts/checkDuplicateFilenames.py
> 
> ```

We’re doing similar things in places, but I’m starting to thing that yaml may not be the best place to write shell scripts?

---

<div class="post-metadata">

**Author:** ![chayden83](https://yyz1.discourse-cdn.com/flex029/user_avatar/discourse.bemanproject.org/chayden83/32/215_2.png) [@chayden83](https://discourse.bemanproject.org/u/chayden83)\
**Post date:** [March 22, 2025, 4:52pm UTC](https://discourse.bemanproject.org/t/using-shell-scripts-in-github-workflow/384/2 "2025-03-22T16:52:02Z")

</div>

I actually think moving script logic out of the CI YAML files is a good idea. I remember when I was learning GitLab CI there were a number of arguments and articles that I found compelling on that front. Ultimately, the strongest argument for me was the testing argument that @Sdowney mentioned.

---

<div class="post-metadata">

**Author:** ![Jeff-Garland](https://yyz1.discourse-cdn.com/flex029/user_avatar/discourse.bemanproject.org/jeff-garland/32/23_2.png) [@Jeff-Garland](https://discourse.bemanproject.org/u/Jeff-Garland)\
**Post date:** [March 24, 2025, 2:08pm UTC](https://discourse.bemanproject.org/t/using-shell-scripts-in-github-workflow/384/3 "2025-03-24T14:08:08Z")

</div>

I think the only problem is we can’t count on things like bash being everywhere – so it’s need to be something like python.

---

<div class="post-metadata">

**Author:** ![Sdowney](https://yyz1.discourse-cdn.com/flex029/user_avatar/discourse.bemanproject.org/sdowney/32/73_2.png) [@Sdowney](https://discourse.bemanproject.org/u/Sdowney)\
**Post date:** [March 24, 2025, 3:44pm UTC](https://discourse.bemanproject.org/t/using-shell-scripts-in-github-workflow/384/4 "2025-03-24T15:44:46Z")

</div>

The script that’s embedded in the yaml is sh or bash now, so it wouldn’t be a regression. I have no objection to python of course. But I think it’s changing from one very technically difficult to run locally implementation for one that is possible for more people?

---

<div class="post-metadata">

**Author:** ![Jeff-Garland](https://yyz1.discourse-cdn.com/flex029/user_avatar/discourse.bemanproject.org/jeff-garland/32/23_2.png) [@Jeff-Garland](https://discourse.bemanproject.org/u/Jeff-Garland)\
**Post date:** [March 25, 2025, 1:48am UTC](https://discourse.bemanproject.org/t/using-shell-scripts-in-github-workflow/384/5 "2025-03-25T01:48:21Z")

</div>

Hmm I wonder if I’m thinking about this wrong – if this is exclusively to run in github land then maybe it’s fine?

---

<div class="post-metadata">

**Author:** ![neatudarius](https://yyz1.discourse-cdn.com/flex029/user_avatar/discourse.bemanproject.org/neatudarius/32/70_2.png) [@neatudarius](https://discourse.bemanproject.org/u/neatudarius)\
**Post date:** [March 25, 2025, 6:50pm UTC](https://discourse.bemanproject.org/t/using-shell-scripts-in-github-workflow/384/6 "2025-03-25T18:50:27Z")

</div>

I would address you other related question: how do we manage to reuse CI flows of not by using scripts?

E.g

- infra repo: ci\_build\_and\_test.sh
- optional: has a simple ci\_build\_and\_test.sh file which runs the script from other repo
- Same for exemplar!

I think we should aime to not have duplicated scripts.

---

<div class="post-metadata">

**Author:** ![Jeff-Garland](https://yyz1.discourse-cdn.com/flex029/user_avatar/discourse.bemanproject.org/jeff-garland/32/23_2.png) [@Jeff-Garland](https://discourse.bemanproject.org/u/Jeff-Garland)\
**Post date:** [March 26, 2025, 2:25pm UTC](https://discourse.bemanproject.org/t/using-shell-scripts-in-github-workflow/384/7 "2025-03-26T14:25:15Z")

</div>

> [@neatudarius](#):
>
> optional: has a simple ci\_build\_and\_test.sh file which runs the script from other repo

I seem to recall @dsankel being much against doing this since the ‘no access to the internet’ installs will not be able to use. But again, if the context here is github workflows I think that doesn’t apply because these aren’t run by hand but only in a github pipeline, right?

---

<div class="post-metadata">

**Author:** ![river](https://yyz1.discourse-cdn.com/flex029/user_avatar/discourse.bemanproject.org/river/32/119_2.png) [@river](https://discourse.bemanproject.org/u/river)\
**Post date:** [March 26, 2025, 3:46pm UTC](https://discourse.bemanproject.org/t/using-shell-scripts-in-github-workflow/384/8 "2025-03-26T15:46:44Z")

</div>

My take is… I think 90% of CI scripts should not be so long that they need to be in a separate file. If it’s give or take \<10 lines, the indirection is not worth it.

GitHub actions are not testable without just triggering the workflow manually or spamming commit at another branch.

So are most bash script.

Which chunks of scripts are you referring to

---

<div class="post-metadata">

**Author:** ![Sdowney](https://yyz1.discourse-cdn.com/flex029/user_avatar/discourse.bemanproject.org/sdowney/32/73_2.png) [@Sdowney](https://discourse.bemanproject.org/u/Sdowney)\
**Post date:** [March 26, 2025, 7:57pm UTC](https://discourse.bemanproject.org/t/using-shell-scripts-in-github-workflow/384/9 "2025-03-26T19:57:12Z")

</div>

This gets repeated more than a few times:

> <https://github.com/bemanproject/optional/blob/main/.github/workflows/ci.yml#L104-L106>

and

> <https://github.com/bemanproject/optional/blob/main/.github/workflows/ci.yml#L156-L171>

is on the edge of what I’d like to factor out, and actually shellcheck before we find some interpolation vulnerability?

Although the first, on the other hand, in practice, I duplicate in `Makefile`. On the gripping hand, DRY.

---

<div class="post-metadata">

**Author:** ![Jeff-Garland](https://yyz1.discourse-cdn.com/flex029/user_avatar/discourse.bemanproject.org/jeff-garland/32/23_2.png) [@Jeff-Garland](https://discourse.bemanproject.org/u/Jeff-Garland)\
**Post date:** [March 26, 2025, 10:08pm UTC](https://discourse.bemanproject.org/t/using-shell-scripts-in-github-workflow/384/10 "2025-03-26T22:08:17Z")

</div>

My $.02 is that is clearly enough to factor out…

---

<div class="post-metadata">

**Author:** ![river](https://yyz1.discourse-cdn.com/flex029/user_avatar/discourse.bemanproject.org/river/32/119_2.png) [@river](https://discourse.bemanproject.org/u/river)\
**Post date:** [March 27, 2025, 10:49pm UTC](https://discourse.bemanproject.org/t/using-shell-scripts-in-github-workflow/384/11 "2025-03-27T22:49:13Z")

</div>

> [@Sdowney](#):
>
> ```auto
> set -x
> cmake --build .build --config Asan --target all_verify_interface_header_sets -- -k 0
> cmake --build .build --config Asan --target all -- -k 0
> 
> ```

I don’t think this is necessary to be put in another file? It’s 3 lines… I don’t think the indirection is worth it.

> [@Sdowney](#):
>
> ```auto
> run: |
> issue_num=$(gh issue list -s open -S "[SCHEDULED-BUILD] Build & Test failure" -L 1 --json number | jq 'if length == 0 then -1 else .[0].number end')
>           
> body=" **Build-and-Test Failure Report**
> - **Time of Failure** : $(date -u '+%B %d, %Y, %H:%M %Z')
> - **Commit** : [${{ github.sha }}](${{ github.server_url }}/${{ github.repository }}/commit/${{ github.sha }})
> - **Action Run** : [View logs](${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }})
>           
> The scheduled build-and-test triggered by cron has failed.
> Please investigate the logs and recent changes associated with this commit or rerun the workflow if you believe this is an error."
>           
> if [[$issue_num -eq -1]]; then
> gh issue create --repo ${{ github.repository }} --title "[SCHEDULED-BUILD] Build & Test failure" --body "$body"
> else
> gh issue comment --repo ${{ github.repository }} $issue_num --body "$body"
> fi
> 
> ```

I agree this should be refactored out. This is universal among all libraries, I can go factor this out as a GitHub Actions Package.

> [@neatudarius](#):
>
> I would address you other related question: how do we manage to reuse CI flows of not by using scripts?
> 
> E.g
> 
> - infra repo: ci\_build\_and\_test.sh
> - optional: has a simple ci\_build\_and\_test.sh file which runs the script from other repo
> - Same for exemplar!
> 
> I think we should aime to not have duplicated scripts.

There’s a more GitHub Action integrated way to do this, see: we can package them as GitHub Actions dependency. I am happy to help with this.