Windows Self-Hosted Test Runners

Introduction

Testable test runners are responsible for executing test cases and reporting back the results.

Windows test runners are available for tests that need to execute on the Windows OS instead of the standard Ubuntu 22.04 LTS inside a container that we use.

Supported scenario types: Selenium, Playwright, Puppeteer, Live sessions, Node.js scripts, HAR replay, Postman, and Java.

These test runners are packaged up as an AMI (i.e. machine image) that you can launch if you use one of the cloud providers we support (AWS, Azure, Google Cloud). See our test runners guide for more details on that.

You can also install and self-hosted our Windows test runner sofware including all dependencies on your own computers.

Prerequisites

To install our test runner it is recommended to spin up a fresh new Windows VM dedicated to being a test runner and not installing it on existing Windows machines. Requirements include:

  • 50GB disk, 2 vCPU, 4GB+ RAM
  • Admin access
  • Windows 10+, Windows Server 2016, Windows Server 2019.

Installation

Steps to install our test runner software on Windows:

  1. Download https://testable-onprem.s3.amazonaws.com/install.cmd
  2. Right click install.cmd and select Run as administrator.
  3. Once it finishes (it may take a while) you should see several new shortcuts on your desktop. Right click the Testable - Configure shortcut and select Run as administrator and follow the instructions to configure and run your test runner.
  4. Once it’s successfully running, go back to the Testable web application under Org Management -> Test Runner Sources -> Self-Hosted and you should see the newly configured test runner under the chosen region name.

Update

On your desktop there should be a shortcut for Testable - Upgrade Runner. Simply right click that and Run as administrator. This will download the latest update, install it, and launch the updated runner.

Automation

In order to automate the installation and startup of the Testable runner you can use the following powershell script as your user data (AWS) or custom data (Azure) or startup script (GCP):

[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::TLS12
Set-ExecutionPolicy -Force bypass

$key = "KEY" # replace with your API Key
$region = "REGIONNAME" # replace with your region name

Write-Host "Setting environment variables"

[System.Environment]::SetEnvironmentVariable("TESTABLE_KEY", "$key", [System.EnvironmentVariableTarget]::Machine)
[System.Environment]::SetEnvironmentVariable("TESTABLE_REGION", "$region", [System.EnvironmentVariableTarget]::Machine)

$env:TESTABLE_KEY="$key"
$env:TESTABLE_REGION="$region"

Write-Host "Installing Testable runner"

$client = (new-object System.Net.WebClient)
$client.DownloadFile(
      "https://testable-onprem.s3.amazonaws.com/install.cmd",
      "install.cmd")

.\install.cmd

Write-Host "Creating Testable startup task"

$Trigger= New-ScheduledTaskTrigger -AtLogon
$Action = New-ScheduledTaskAction -Execute "C:\TestableAgent\Runner\startup.onprem.cmd"
Register-ScheduledTask -TaskName "Testable" -Trigger $Trigger -User "$env:UserName" -Action $Action

Write-Host "Setting up registry for Testable autologon"

net user $env:UserName '$(%YuTiHeKHTq!!=*;FGzOhptu.qJ(BT'
$RegPath = "HKLM:\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Winlogon"
Set-ItemProperty -Path $RegPath -Name "AutoAdminLogon" -Value "1"
Set-ItemProperty -Path $RegPath -Name "DefaultUsername" -Value "$env:UserName"
Set-ItemProperty -Path $RegPath -Name "DefaultPassword" -Value '$(%YuTiHeKHTq!!=*;FGzOhptu.qJ(BT'

Write-Host "Restarting computer"

Restart-Computer -Force

Make sure to set the $key and $region variables as appropriate for your account. You can update the test runner any time as indicated above.