Windows Self-Hosted Test Runners

Introduction

Testable test runners are responsible for executing test cases and reporting back the results.

Windows test runners are available for tests that need to execute on the Windows OS instead of the standard Ubuntu 22.04 LTS inside a container that we use.

Supported scenario types: Selenium, Playwright, Puppeteer, Live sessions, Node.js scripts, HAR replay, Postman, and Java.

These test runners are packaged up as an AMI (i.e. machine image) that you can launch if you use one of the cloud providers we support (AWS, Azure, Google Cloud). See our test runners guide for more details on that.

You can also install and self-hosted our Windows test runner sofware including all dependencies on your own computers.

Prerequisites

To install our test runner it is recommended to spin up a fresh new Windows VM dedicated to being a test runner and not installing it on existing Windows machines. Requirements include:

  • 50GB disk, 2 vCPU, 4GB+ RAM
  • Admin access
  • Windows 10+, Windows Server 2016, Windows Server 2019.

Installation

Steps to install our test runner software on Windows:

  1. Download https://testable-onprem.s3.amazonaws.com/install.cmd
  2. Right click install.cmd and select Run as administrator.
  3. Once it finishes (it may take a while) you should see several new shortcuts on your desktop. Right click the Testable - Configure shortcut and select Run as administrator and follow the instructions to configure and run your test runner.
  4. Once it’s successfully running, go back to the Testable web application under Org Management -> Test Runner Sources -> Self-Hosted and you should see the newly configured test runner under the chosen region name.

Update

On your desktop there should be a shortcut for Testable - Upgrade Runner. Simply right click that and Run as administrator. This will download the latest update, install it, and launch the updated runner.