Windows Self-Hosted Test Runners

Introduction

Testable test runners are responsible for executing test cases and reporting back the results.

Windows test runners are available for tests that need to execute on the Windows OS instead of the standard Ubuntu 22.04 LTS inside a container that we use.

Supported scenario types: Selenium, Playwright, Puppeteer, Live sessions, Node.js scripts, HAR replay, Postman, and Java.

These test runners are packaged up as an AMI (i.e. machine image) that you can launch if you use one of the cloud providers we support (AWS, Azure, Google Cloud). See our test runners guide for more details on that.

You can also install and self-hosted our Windows test runner sofware including all dependencies on your own computers.

Prerequisites

To install our test runner it is recommended to spin up a fresh new Windows VM dedicated to being a test runner and not installing it on existing Windows machines. Requirements include:

  • 50GB disk, 2 vCPU, 4GB+ RAM
  • Admin access
  • Windows 10+, Windows Server 2016, Windows Server 2019.

Installation

Steps to install our test runner software on Windows:

  1. Download https://testable-onprem.s3.amazonaws.com/testable-windows-installer-latest.zip
  2. Extract the zip file
  3. In a file explorer navigate to the extracted folder, right click run.cmd and select Run as administrator.
  4. Once it finishes (it may take a while) you should see several new shortcuts on your desktop. Right click the Testable - Configure shortcut and select Run as administrator and follow the instructions to configure and run your test runner.
  5. Once it’s successfully running, go back to the Testable web application under Org Management -> Test Runner Sources -> Self-Hosted and you should see the newly configured test runner under the chosen region name.

Update

In order to update your test runner to the latest version post-install follow these steps:

  1. Download https://testable-onprem.s3.amazonaws.com/testable-windows-minimal-updater-latest.zip
  2. Extract the zip file
  3. In a file explorer navigate to the extracted folder, right click run-update.cmd and select Run as administrator.

This will update the test runner agent code. Once done, restart the test runner using the shortcuts on the desktop to pick up the latest changes.