Configuring CI Using GitHub Actions and Nx
There are two general approaches to setting up CI with Nx - using a single pipeline or using distributed task execution. For smaller repositories, a single pipeline is faster and cheaper, but once a full CI run starts taking 10 to 15 minutes, distributed task execution becomes the better option. Distributed task execution allows you to keep the CI pipeline fast as you scale. As the repository grows, all you need to do is add more agents.
Single Pipeline
This is an example of a GitHub Actions setup that runs on a single pipeline, building and testing only what is affected.
1name: CI
2on:
3 push:
4 branches:
5 # Change this if your primary branch is not main
6 - main
7 pull_request:
8
9jobs:
10 main:
11 runs-on: ubuntu-latest
12 steps:
13 - uses: actions/checkout@v4
14 with:
15 fetch-depth: 0
16 # Cache node_modules
17 - uses: actions/setup-node@v3
18 with:
19 node-version: 20
20 cache: 'npm'
21 - run: npm ci
22 - uses: nrwl/nx-set-shas@v3
23 # This line is needed for nx affected to work when CI is running on a PR
24 - run: git branch --track main origin/main
25
26 - run: npx nx format:check
27 - run: npx nx affected -t lint,test,build --parallel=3
28GitHub can track the last successful run on the main branch and use this as a reference point for the BASE. The nrwl/nx-set-shas provides a convenient implementation of this functionality which you can drop into your existing CI config. To understand why knowing the last successful build is important for the affected command, check out the in-depth explanation in Actions's docs.
Distributed Task Execution
To set up Distributed Task Execution (DTE), you can run this generator:
❯
npx nx g ci-workflow --ci=github
Or you can copy and paste the workflow below:
1name: CI
2on:
3 push:
4 branches:
5 - main
6 pull_request:
7
8jobs:
9 main:
10 name: Nx Cloud - Main Job
11 uses: nrwl/ci/.github/workflows/nx-cloud-main.yml@v0.13.0
12 with:
13 number-of-agents: 3
14 parallel-commands: |
15 npx nx-cloud record -- npx nx format:check
16 parallel-commands-on-agents: |
17 npx nx affected -t lint,test,build --parallel=2
18
19 agents:
20 name: Nx Cloud - Agents
21 uses: nrwl/ci/.github/workflows/nx-cloud-agents.yml@v0.13.0
22 with:
23 number-of-agents: 3
24This configuration is using two reusable workflows from the nrwl/ci repository. You can check out the full API for those workflows.
The first workflow is for the main job:
1 uses: nrwl/ci/.github/workflows/nx-cloud-main.yml@v0.13.0
2The parallel-commands script will be run on the main job. The parallel-commands-on-agents script will be distributed across the available agents.
The second workflow is for the agents:
1 uses: nrwl/ci/.github/workflows/nx-cloud-agents.yml@v0.13.0
2The number-of-agents property controls how many agent jobs are created. Note that this property should be the same number for each workflow.
The number-of-agents property and the --parallel flag both parallelize tasks, but in different ways. The way this workflow is written, there will 3 agents running tasks and each agent will try to run 2 tasks at once. If a particular CI run only has 2 tasks, only one agent will be used.
Custom Distributed CI with Nx Cloud
Our reusable GitHub workflow represents a good set of defaults that works for a large number of our users. However, reusable GitHub workflows come with their limitations.
If the reusable workflow above doesn't satisfy your needs you should create a custom workflow. If you were to rewrite the reusable workflow yourself, it would look something like this:
1name: CI
2on:
3 push:
4 branches:
5 - main
6 pull_request:
7
8env:
9 NX_CLOUD_DISTRIBUTED_EXECUTION: true # this enables DTE
10 NX_CLOUD_DISTRIBUTED_EXECUTION_AGENT_COUNT: 3 # expected number of agents
11 NX_BRANCH: ${{ github.event.number || github.ref_name }}
12 NX_CLOUD_ACCESS_TOKEN: ${{ secrets.NX_CLOUD_ACCESS_TOKEN }}
13 NPM_TOKEN: ${{ secrets.NPM_TOKEN }} # this is needed if our pipeline publishes to npm
14
15jobs:
16 main:
17 name: Nx Cloud - Main Job
18 runs-on: ubuntu-latest
19 steps:
20 - uses: actions/checkout@v4
21 name: Checkout [Pull Request]
22 if: ${{ github.event_name == 'pull_request' }}
23 with:
24 # By default, PRs will be checked-out based on the Merge Commit, but we want the actual branch HEAD.
25 ref: ${{ github.event.pull_request.head.sha }}
26 # We need to fetch all branches and commits so that Nx affected has a base to compare against.
27 fetch-depth: 0
28
29 - uses: actions/checkout@v4
30 name: Checkout [Default Branch]
31 if: ${{ github.event_name != 'pull_request' }}
32 with:
33 # We need to fetch all branches and commits so that Nx affected has a base to compare against.
34 fetch-depth: 0
35
36 # Set node/npm/yarn versions using volta
37 - uses: volta-cli/action@v4
38 with:
39 package-json-path: '${{ github.workspace }}/package.json'
40
41 - name: Use the package manager cache if available
42 uses: actions/setup-node@v3
43 with:
44 node-version: 20
45 cache: 'npm'
46
47 - name: Install dependencies
48 run: npm ci
49
50 - name: Check out the default branch
51 run: git branch --track main origin/main
52
53 - name: Initialize the Nx Cloud distributed CI run
54 run: npx nx-cloud start-ci-run
55
56 - name: Run commands in parallel
57 run: |
58 pids=()
59 # list of commands to be run on main has env flag NX_CLOUD_DISTRIBUTED_EXECUTION set to false
60 NX_CLOUD_DISTRIBUTED_EXECUTION=false npx nx-cloud record -- npx nx format:check & pids+=($!)
61
62 # list of commands to be run on agents
63 npx nx affected -t lint,test,build --parallel=3 &
64 pids+=($!)
65
66 # run all commands in parallel and bail if one of them fails
67 for pid in ${pids[*]}; do
68 if ! wait $pid; then
69 exit 1
70 fi
71 done
72
73 exit 0
74
75 - name: Stop all running agents for this CI run
76 # It's important that we always run this step, otherwise in the case of any failures in preceding non-Nx steps, the agents will keep running and waste billable minutes
77 if: ${{ always() }}
78 run: npx nx-cloud stop-all-agents
79
80 agents:
81 name: Agent ${{ matrix.agent }}
82 runs-on: ubuntu-latest
83 strategy:
84 matrix:
85 # Add more agents here as your repository expands
86 agent: [1, 2, 3]
87 steps:
88 - name: Checkout
89 uses: actions/checkout@v4
90
91 # Set node/npm/yarn versions using volta
92 - uses: volta-cli/action@v4
93 with:
94 package-json-path: '${{ github.workspace }}/package.json'
95
96 - name: Use the package manager cache if available
97 uses: actions/setup-node@v3
98 with:
99 node-version: 20
100 cache: 'npm'
101
102 - name: Install dependencies
103 run: npm ci
104
105 - name: Start Nx Agent ${{ matrix.agent }}
106 run: npx nx-cloud start-agent
107 env:
108 NX_AGENT_NAME: ${{ matrix.agent }}
109There are comments throughout the workflow to help you understand what is happening in each section.