Running tests using Github Actions

Using Github Actions, you can setup a CI/CD pipeline that automatically runs UI-licious tests, whenever there is new code pushed to your repository, or perhaps at a scheduled interval.

Run tests when commits are pushed to target branches and get immediate feedback when something is broken.

This section will show you how to setup a workflow to run a suite of UI-licious tests on push.

To see a completed example, you can check out this Github projectopen in new window for reference.

First, create the .github/workflows folder in your repository.

Then, create run-uilicious-tests.yml file in the .github/workflows folder.

Copy and paste the following to the run-uilicious-tests.yml file:

name: Run UI-licious Tests
run-name: Run UI-licious Tests
  # run uilicious tests when commits are pushed to target branches
      - 'main'
  # The "sync-to-cloud" job:
  # This sync all test files from the test directory in the repository to UI-licious Cloud
    runs-on: ubuntu-latest
      - name: Checkout HEAD
        uses: actions/checkout@v3
          ref: ${{ github.head_ref }}
      - name: Install uilicious-cli
        run: npm install -g uilicious-cli
      - name: Upload UI-licious Project
        run: uilicious-cli upload ${{ vars.UILICIOUS_PROJECT_ID }} ${{ vars.UILICIOUS_TEST_DIR }} --key ${{ secrets.UILICIOUS_ACCESS_KEY }}
  # The "run-test" job:
  # This runs specified test files in parallel
    # Run 'sync-to-cloud' job is complete.
    needs: sync-to-cloud 
        # List of test files to run
        test_file_path: [
        # List of browsers to run the tests on
        browser: ['chrome']
        # List of resolutions to run the tests on
        resolution: [{width: 1280, height: 800}]
      # Number of Github Runners to run this job in parallel
      # - This should not be more than the number of UI-licious test runners that you have available
      max-parallel: 2 
    runs-on: ubuntu-latest
      - run: echo '"${{ matrix.test_file_path }}" (${{ matrix.browser }} ${{ matrix.resolution.width }}x${{ matrix.resolution.height }})'
      - name: Install uilicious-cli
        run: npm install -g uilicious-cli
      - name: Run File
        run: uilicious-cli run ${{ vars.UILICIOUS_PROJECT_ID }} "${{ matrix.test_file_path }}" --browser ${{ matrix.browser }} --width ${{ matrix.resolution.width }} --height ${{ matrix.resolution.height }} --key ${{ secrets.UILICIOUS_ACCESS_KEY }}

This workflow will trigger when changes are pushed to the main branch. You can change the conditions to trigger workflow at lines 4 - 6. See Github's documentation on the pushopen in new window trigger.

This workflow runs two jobs:

  1. sync-to-cloud: This job uploads all tests found in the UILICIOUS_TEST_DIR to the UI-licious Cloud. If your team only uses the Editor interface on UI-licious Cloud for writing tests, then this job is not neccessary, and you can remove this job entirely and needs: sync-to-cloud at line 22.
  2. run-test: This job runs a suite of test based on the matrix configuration.

Configuring the tests you want to run under matrix in the run-test job:

  • test_file_path: The list of tests you want to run
  • browser: The browser(s) you want to run.
  • resolution The resolution(s) you want to run the test on.

Github actions will create a job for each combination of test_file_path x browser x resolution, and run them in parallel.

Configure the number of parallel runs by changing max-parallel under the run-test job. Note that this should not be more than the number of UI-licious test runners you have available in your UI-licious subscription.

Configure the following Github Actions variables:

  • UILICIOUS_PROJECT_ID : This is the UI-licious project ID, which you can find in the "Project Settings" page.
  • UILICIOUS_TEST_DIR : Which folder in the respository do save the UI-licious files to? E.g. "./tests". This is not needed the sync-to-cloud job is not necessary for your process.

Configure the following Github Actions secrets:

Finally, commit .github/workflow/run-uilicious-tests.yml to the your repository, and make a change on your target branch to trigger the workflow!

Last Updated: