posts: more updates to github tips and tricks [staging]

Signed-off-by: Harsh Shandilya <me@msfjarvis.dev>
This commit is contained in:
Harsh Shandilya 2021-01-03 11:05:34 +05:30
parent 217c72caf5
commit 5ea57261ae
2 changed files with 23 additions and 19 deletions

View File

@ -9,8 +9,6 @@ tags = ["tips and tricks", "github actions", "schedules", "jobs", "workflows"]
title = "Tips and Tricks for GitHub Actions"
+++
> // TODO: add more links across the entire post.
GitHub Actions has grown at a rapid pace, and has become the CI platform of choice for most open source projects. The recent changes to Travis CI's pricing for open source is certainly bound to accelerate this even more.
Due to it being a first-party addition to GitHub, Actions has nearly infinite potential to run jobs in reaction to changes on GitHub. You can automatically set labels to newly opened pull requests, greet first time contributors, and more.
@ -19,7 +17,7 @@ Let's go over some things that you can do with Actions, and we'll end it with so
## Running workflows based on a cron trigger
For [Android Password Store](https://msfjarvis.dev/aps), we maintain a list of known [publicsuffixes](https://publicsuffix.org/) to be able to do efficient detection of the 'base' domain of a website we're autofilling into. This list changes frequently, and we typically sync our repository with the latest copy on a weekly basis. Actions enables us to do this automatically, like this:
For [Android Password Store](https://msfjarvis.dev/aps), we maintain a list of known [public suffixes](https://publicsuffix.org/) to be able to do efficient detection of the 'base' domain of the website we're autofilling into. This list changes frequently, and we typically sync our repository with the latest copy on a weekly basis. Actions enables us to do this automatically, like this:
```yaml
name: Update Publix Suffix List data
@ -38,7 +36,7 @@ Mine is a very naive example of how you can use cron triggers to automate parts
## Running jobs based on commit message
Continuous delivery is great, but sometimes you want slightly more control. Rather than run a deployment task on each push to your repository, what if you want it to only run when a specific keyword is in the commit message? Actions has support for this natively, and the deployment pipeline of this very site relies on this feature.
Continuous delivery is great, but sometimes you want slightly more control. Rather than run a deployment task on each push to your repository, what if you want it to only run when a specific keyword is in the commit message? Actions has support for this natively, and the deployment pipeline of this very site relies on this feature:
```yaml
name: Deploy to Cloudflare Workers Sites
@ -60,9 +58,11 @@ jobs:
This snippet defines a job that is only executed when the top commit of the push contains the text `[deploy]` in its message, and another that only runs when the commit message contains `[staging]`. Together, these let me control if I want a change to not be immediately deployed, deployed to either the main or staging site, or to both at the same time. So now I can update a draft post without a full re-deployment of the main site, or make a quick edit to a published post that doesn't need to be reflected in the staging environment.
The core logic of this operation is composed of three parts. The [github context](https://docs.github.com/en/free-pro-team@latest/actions/reference/context-and-expression-syntax-for-github-actions#github-context), the [if conditional](https://docs.github.com/en/free-pro-team@latest/actions/reference/workflow-syntax-for-github-actions#jobsjob_idif) and the [contains](https://docs.github.com/en/free-pro-team@latest/actions/reference/context-and-expression-syntax-for-github-actions#contains) method. The linked documentation for each does a great job at explaining them, and has further references to allow you to fulfill even more advanced use cases.
## Testing across multiple configurations in parallel
GitHub also comes with an amazing matrix functionality that lets you define a set of variables that are then used to construct different combinations to cover all cases. As you can tell by this confusing explanation, I failed math every single time it was a subject, so here's a table to explain the same thing and then we'll see how to use this in GitHub Actions.
Jobs in a workflow run in parallel by default, and GitHub comes with an amazing matrix functionality that can automatically generate multiple jobs for you from a single definition. Take this specific example:
| | Windows | MacOS | Ubuntu |
|---------|-------------------|-----------------|------------------|
@ -72,13 +72,11 @@ GitHub also comes with an amazing matrix functionality that lets you define a se
> {{< sub "This particular matrix is for Rust, to test a codebase across Windows, Ubuntu, and macOS using the Rust stable, beta, and nightly toolchains." >}}
In GitHub Actions, we can simply provide the platforms (Windows, MacOS and, Ubuntu) and the Rust channels (Stable, Beta, and Nightly) and have it figure out how to make the table above. Here's how we do it:
In GitHub Actions, we can simply provide the platforms (Windows, MacOS and, Ubuntu) and the Rust channels (Stable, Beta, and Nightly) inside a single job and let it figure out how to make the permutations and create separate jobs for them. To configure such a matrix, we write something like this:
```yaml
jobs:
check-rust-code:
# Make the job run on the matrix' current OS
runs-on: ${{ matrix.os }}
strategy:
# Defines a matrix strategy
matrix:
@ -86,23 +84,25 @@ jobs:
os: [ubuntu-latest, windows-latest, macOS-latest]
# Sets the Rust channels we want to test against
rust: [stable, beta, nightly]
# Make the job run on the OS picked by the matrix
runs-on: ${{ matrix.os }}
steps:
- uses: actions-rs/toolchain@v1
with:
profile: minimal
components: rustfmt, clippy
# Installs the Rust toolchain for the matrix' current Rust channel
# Installs the Rust toolchain for the channel picked by the matrix
toolchain: ${{ matrix.rust }}
```
This will automatically generate 9 (3 platforms * 3 Rust channels) parallel jobs to test this entire configuration.
This will automatically generate 9 (3 platforms * 3 Rust channels) parallel jobs to test this entire configuration, without requiring us to manually define them. [DRY](https://en.wikipedia.org/wiki/Don%27t_repeat_yourself) at its finest :)
## Make a job run after another job
## Make a job run after another
Since you can define more than one job in one Actions workflow, you can also leverage that functionality to define more complex requirements for their execution. More real world examples!
By default, jobs defined in a workflow file run in parallel. However, we might need a more sequential order of execution for some cases, and GHA does include support for this case. Let's try another real world example:.
LeakCanary has a checks job that runs on each push to the main branch and on each pull request, and they wanted to add support for snapshot deployment to finally retire Travis CI. To make this happen, I simply added a new job to the same workflow, and made it run only on push events. For added guarantees, I also made the deployment job have a dependency on the checks job. This ensures that there won't be a snapshot deployment until all tests are passing on main. The relevant parts of the workflow configuration are here:
[LeakCanary](https://github.com/square/leakcanary) has a [checks job](https://github.com/square/leakcanary/blob/f5343aca6e019994f7e69a28fac14ca18e071b88/.github/workflows/main.yml) that runs on each push to the main branch and on each pull request. They wanted to add support for snapshot deployment, in order to finally retire Travis CI. To make this happen, I simply added a [new job](https://github.com/square/leakcanary/pull/2044/commits/a6f6c204559396120836b27c0b2a46d3e444c728) to the same workflow, and made it run only on push events. For added confidence, I also made the deployment job have a dependency on the checks job, which ensures that there won't be a snapshot deployment until all tests are passing on main. The relevant parts of the workflow configuration are here:
```yaml
on:
@ -122,19 +122,23 @@ jobs:
needs: [checks]
```
# Mitigating security concerns wih Actions
# Mitigating security concerns with Actions
GitHub Actions benefits from a vibrant ecosystem of user-authored actions, which opens it up to equal opportunities for abuse. It is relatively easy to work around the common ones, and I'm going to outline them here. I'm no authority on security, and these recommendations are based on a combination of my reading and understanding. These *should* be helpful, but this list is not exhaustive, and you should exercise all the caution you can.
## Use exact commit hashes rather than tags
Tags are moving qualifiers, and can be force pushed at any moment. If the repository for an Action you use in your workflows is compromised, the tag you use could be force pushed with a malicious version that exfiltrates secrets to a third-party server. Auditing the source of a repository at a given tag, then using the SHA1 commit hash it currently points to as the version alleviates that concern due to it being extremely difficult to fake a new commit with the exact hash.
Tags are moving qualifiers, and can be [force pushed at any moment](https://julienrenaux.fr/2019/12/20/github-actions-security-risk/). If the repository for an Action you use in your workflows is compromised, the tag you use could be force pushed with a malicious version that exfiltrates secrets to a third-party server. Auditing the source of a repository at a given tag, then using the SHA1 commit hash it currently points to as the version alleviates that concern due to it being extremely difficult to fake a new commit with the exact hash.
To get the commit hash for a specific tag, head to the Releases page of the repository, then click the short SHA1 hash below the tag name and copy the full hash from the URL.
> // TBD: image
![A tag along with its commit hash](/uploads/actions_tips_tricks_commit_hash.webp)
An alternative to this approach is to vendor each third-party action you use into your own repository, and then use the local copy as the source. This puts you in charge of manually syncing to each version, but allows you to use the safer "Only allow actions from within this repository" option in your repository's Actions settings. Having to manually sync with each release also gives you slightly better visibility into the changes between versions since they'd be available in a single PR diff.
> {{< sub "Here, the commit hash is feb985e. Ideally, you want to click that link and copy the full hash from the URL" >}}
### Alternate solution
A more extreme fix for this problem is to [vendor](https://stackoverflow.com/questions/26217488/what-is-vendoring) each third-party action you use into your own repository, and then use the local copy as the source. This puts you in charge of manually syncing the source to each version, but allows you to restrict the allowed Actions to ones in your repository thereby greatly increasing security. However, having to manually sync can get tedious if your workflows involve a lot of third-party actions. However, the same manual sync also gives you slightly better visibility into the changes between versions since they'd be available in a single PR diff.
To use an Action from a local directory, replace the `uses:` line with the relative path to the local copy in the repository.
@ -149,7 +153,7 @@ job:
## Replace `pull_request_target` with `pull_request`
`pull_request_target` grants a PR access to a github token that can write to your repository, exposing your code to modification by a malicious third-party who simply needs to open a PR against your repository. Most people will already be on `pull_request`, but if you are not, audit your requirements and make the switch.
`pull_request_target` grants a PR access to a github token that can write to your repository, exposing your code to modification by a malicious third-party who simply needs to open a PR against your repository. Most people will already be using the safe `pull_request` event, but if you are not, audit your requirements for `pull_request_target` and make the switch.
```diff
-on: [push, pull_request_target]
@ -158,4 +162,4 @@ job:
{{< horizontal_line >}}
And that's the end of this far too long post. If you read it all, I thank you for your patience. I'm still learning about Actions, and if you have a similar trick that I did not cover here, I'd love to hear about it! Comment below or just tweet at me on [@msfjarvis](https://twitter.com/msfjarvis) :)
I'm still learning about Actions, and there is a lot that I did not cover here. I highly encourage readers to refer the GitHub docs for [Workflow syntax](https://docs.github.com/en/free-pro-team@latest/actions/reference/workflow-syntax-for-github-actions) and [Context and expressions syntax](https://docs.github.com/en/free-pro-team@latest/actions/reference/context-and-expression-syntax-for-github-actions) to gain more knowledge of the workflow configuration capabilities. Let me know if you find something cool that I did not cover here!

Binary file not shown.

After

Width:  |  Height:  |  Size: 13 KiB