Running Our Tests in Parallel with Cypress Dashboard

In this lesson, we will learn how to integrate the Cypress Dashboard with GitHub Actions so that our tests will run in parallel. The Dashboard will also provide us with greater insights into our tests which will come in handy anytime our tests fail.

You will need to signup for a free account over at cypress.io/dashboard. We recommend using your GitHub account to register.

After creating your account and logging in, you will need to create your first project.

screencapture-dashboard-cypress-io-organizations-c11ae021-28cd-4e97-922d-f6bca2ceaf3b-projects-new-2022-05-09-14_34_39.png

Give your project a name and select GitHub Actions as the CI provider.

screencapture-dashboard-cypress-io-organizations-c11ae021-28cd-4e97-922d-f6bca2ceaf3b-projects-new-2022-05-09-14_35_23.png

Next, we will need to copy the projectID into our cypress.json file.

screencapture-dashboard-cypress-io-projects-r5p488-runs-2022-05-09-14_35_47.png

{
  "baseUrl": "http://localhost:3000",
  "viewportHeight": 1000,
  "viewportWidth": 1280,
  "projectId": "r5p488"
}

Screen Shot 2022-05-09 at 2.36.52 PM.png

Then we will need to copy the record key and add it to GitHub secrets.

screencapture-dashboard-cypress-io-projects-r5p488-runs-2022-05-09-14_38_10.png

screencapture-github-robertguss-vercel-commerce-settings-secrets-actions-new-2022-05-09-14_39_27.png

screencapture-github-robertguss-vercel-commerce-settings-secrets-actions-2022-05-09-14_39_56.png

Finally, we will need to add some environment variables to the GitHub Actions config file.

CYPRESS_RECORD_KEY: ${{ secrets.CYPRESS_RECORD_KEY }}
# pass GitHub token to allow accurately detecting a build vs a re-run build
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

We also need to add record: true underneath the wait-on: directive. This will tell the dashboard to record our test runs.

The entire GitHub Actions workflow file should look like this:

name: E2E on Chrome

on: [push]

jobs:
  install:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v2

      - name: Cypress run
        uses: cypress-io/github-action@v3
        with:
          project: ./site
          browser: chrome
          build: yarn build
          start: yarn start
          wait-on: "http://localhost:3000"
          record: true
        env:
          COMMERCE_PROVIDER: ${{ secrets.COMMERCE_PROVIDER }}
          NEXT_PUBLIC_SHOPIFY_STOREFRONT_ACCESS_TOKEN: ${{ secrets.NEXT_PUBLIC_SHOPIFY_STOREFRONT_ACCESS_TOKEN }}
          NEXT_PUBLIC_SHOPIFY_STORE_DOMAIN: ${{ secrets.NEXT_PUBLIC_SHOPIFY_STORE_DOMAIN }}
          CYPRESS_RECORD_KEY: ${{ secrets.CYPRESS_RECORD_KEY }}
          # pass GitHub token to allow accurately detecting a build vs a re-run build
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

Note that we are passing the GITHUB_TOKEN as well. This does not need to be in our GitHub secrets, GitHub will handle this for us. It is used by the dashboard to detect a build vs a re-run build.

Save your changes and push them up to GitHub.

git add .
git commit -m "added Cypress dashboard projectID and env vars to GHA config"
git push origin github-actions-cypress-tests

If you check the Cypress dashboard you should see the latest runs.

Screen Shot 2021-12-17 at 9.41.53 AM.png

After clicking on it, you can then inspect the test results.

Screen Shot 2021-12-17 at 9.42.36 AM.png

Now that our tests are hooked up to the dashboard we can easily inspect each test whenever there is a failure. The dashboard will provide a screenshot of the exact point of failure and even a video that we can watch to see exactly where and why our tests failed.

Running our Tests in Parallel

In order to have our tests run in parallel, we have to make some changes to our GitHub Actions config file.

Copy and paste the following into .github/workflows/main.yml

name: E2E on Chrome

on: [push]

jobs:
  install:
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        containers: [1, 2, 3]
    steps:
      - name: Checkout
        uses: actions/checkout@v2

      - name: Cypress run
        uses: cypress-io/github-action@v3
        with:
          project: ./site
          browser: chrome
          build: yarn build
          start: yarn start
          wait-on: "http://localhost:3000"
          record: true
          parallel: true
        env:
          COMMERCE_PROVIDER: ${{ secrets.COMMERCE_PROVIDER }}
          NEXT_PUBLIC_SHOPIFY_STOREFRONT_ACCESS_TOKEN: ${{ secrets.NEXT_PUBLIC_SHOPIFY_STOREFRONT_ACCESS_TOKEN }}
          NEXT_PUBLIC_SHOPIFY_STORE_DOMAIN: ${{ secrets.NEXT_PUBLIC_SHOPIFY_STORE_DOMAIN }}
          CYPRESS_RECORD_KEY: ${{ secrets.CYPRESS_RECORD_KEY }}
          # pass GitHub token to allow accurately detecting a build vs a re-run build
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

The updated install job is similar to what we had previously, with some important modifications. Let’s review all of the steps of this job in order.

First, we give our job a name, “E2E on Chrome.”

Then, we tell GitHub actions to only trigger this workflow on: [push]

Next, we tell GitHub Actions to run this job on the latest version of Ubuntu.

name: E2E on Chrome

on: [push]

jobs:
  install:
    runs-on: ubuntu-latest

Next, we tell GitHub Actions our strategy for parallelization. The fail-fast: false will prevent other machines running in parallel from being canceled if one of our tests fails. If one of our tests fails in one of the machines, we do not want the other machines currently running in parallel to be canceled because of it. Only the machine with the failing test should be canceled.

The matrix directive tells GitHub how many containers we would like to use, which in this case is 3. So, GitHub will spin up a maximum of 3 containers to run our tests in parallel.

name: E2E on Chrome

on: [push]

jobs:
  install:
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        containers: [1, 2, 3]

Next, we use the GitHub Checkout action again, to check out our repo’s code in each container.

name: E2E on Chrome

on: [push]

jobs:
  install:
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        containers: [1, 2, 3]
    steps:
      - name: Checkout
        uses: actions/checkout@v2

Then we use the official Cypress GitHub Action.

name: E2E on Chrome

on: [push]

jobs:
  install:
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        containers: [1, 2, 3]
    steps:
      - name: Checkout
        uses: actions/checkout@v2

      - name: Cypress run
        uses: cypress-io/github-action@v3
        with:
          project: ./site
          browser: chrome
          build: yarn build
          start: yarn start
          wait-on: "http://localhost:3000"
          record: true
          parallel: true

Notice that we have added parallel: true. This tells the dashboard we want our tests to be run in parallel.

Finally, we add our environment variables and the entire workflow config should look like the following.

name: E2E on Chrome

on: [push]

jobs:
  install:
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        containers: [1, 2, 3]
    steps:
      - name: Checkout
        uses: actions/checkout@v2

      - name: Cypress run
        uses: cypress-io/github-action@v3
        with:
          project: ./site
          browser: chrome
          build: yarn build
          start: yarn start
          wait-on: "http://localhost:3000"
          record: true
          parallel: true
        env:
          COMMERCE_PROVIDER: ${{ secrets.COMMERCE_PROVIDER }}
          NEXT_PUBLIC_SHOPIFY_STOREFRONT_ACCESS_TOKEN: ${{ secrets.NEXT_PUBLIC_SHOPIFY_STOREFRONT_ACCESS_TOKEN }}
          NEXT_PUBLIC_SHOPIFY_STORE_DOMAIN: ${{ secrets.NEXT_PUBLIC_SHOPIFY_STORE_DOMAIN }}
          CYPRESS_RECORD_KEY: ${{ secrets.CYPRESS_RECORD_KEY }}
          # pass GitHub token to allow accurately detecting a build vs a re-run build
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

Let’s push up these latest changes to see our tests run in parallel.

git add .
git commit -m "added parallelization to GHA config"
git push origin github-actions-cypress-tests

Now that we have split up our workflow into multiple jobs, the install job will run first.

Screen Shot 2021-12-17 at 10.09.51 AM.png

Once the install job is complete our tests will be run in parallel on multiple machines.

Screen Shot 2021-12-17 at 10.13.33 AM.png

All of our tests are passing and finished in a fraction of the time since we have them running in parallel now.

Screen Shot 2021-12-17 at 1.36.55 PM.png

Real World App

rwa-readme-screenshot.png

If you would like to see how the dashboard integrates with a real-world application, check out the Real World App created by the Cypress team. The Real World App (RWA) is a payment application that demonstrates real-world usage of Cypress testing methods, patterns, and workflows. The Cypress dashboard for this application is open to the public so feel free to inspect any PR to see how it integrates with the Dashboard.

https://github.com/cypress-io/cypress-realworld-app

Summary

In this lesson, we learned how to send our tests to the Cypress Dashboard. Then we learned how to configure our GitHub Action to run our tests in parallel with the Dashboard.