Billing

Plan Types

Testable has a variety of different types of plans that can be mixed and matched to best fit your team’s needs.

The general plan categories include:

  • Functional Testing - By Browser Automation Session: Choose a plan which lets you run a number of concurrent browser sessions. These sessions can utilize our remote testing capabilities (e.g. Selenium grid, remote browser grid via CDP, or remote Playwright grid) or upload and execute test scripts directly on Testable test runners.
  • Functional Testing - By Live Session: A live session spins up a browser according to your requirements and lets you run a test manually on it. These plans let you run a number of concurrent live sessions.
  • Performance Testing - By Virtual User: Our simplest performance testing option. Simply pay based on how many virtual users your test generates.
  • Performance Testing - By Test Runner: Read below for more details on what a test runner is. This type of plan tends to be for more advanced teams that want to control exactly how many VMs there test runs across and how much CPU/memory to assign each virtual user. We have several plans that use test runners as the unit of billing.
  • Monitoring: These plans bill based on how many monitoring checks you run.

Plan Options

Testable plans are extremely flexible along a variety of dimensions:

  • Testable Hosted vs Self-Hosted: Your team can save if you choose a plan that runs your tests on self-hosted test runners. This does not mean you have to maintain a grid of test runner VMs. Testable can spin them up as necessary to run your tests. If you use self-hosted test runners but have a plan with Testable hosted capacity you will receive a 10% discount on the billed quantity.
  • By Hour/Minute vs Unlimited: Some plans give you a certain capacity with no limit on the number of tests (e.g. 2 browser automation sessions, 100 unlimited virtual users). This makes sense for teams who want to utilize our platform frequently as part of their test automation. You can also pay by hour or minute (e.g. virtual user hours, monitoring minutes) if your team runs less frequent tests.
  • Annual vs Monthly vs Pay as you go: Choose whether your plan renews annually (cheapest), monthly, or to pay as you go (most expensive). Depending on how regularly you test or how long you plan to use our platform will guide your choice here.

Plan Features

For a full list of plan features for each plan login to Testable, go to Org Management => Billing, and press the Buy Plan button. Some common plan features include:

  • Queueing: If your test cannot be run right now within your purchased plans, but can be run later, the test will queue until enough plan capacity exists to run it later.
  • Unlimited Team Members: Grant as many team members as you want access to your Testable organizations. There is no charge for extra team members.
  • Unlimited Concurrent Users: For performance testing, we do not limit how many concurrent virtual users you can run within your purchased capacity. So for example if you purchase 100k virtual user hours you can choose to run 100 1 hour tests with 1k virtual users or a single 1 hour test with 100k virtual users.
  • Unlimited Tests: Within your purchased capacity run as many tests as you want.
  • Rollover: Our plans that include a consumable resource (e.g. 100 virtual user hours) will automatically rollover any unused quantity to the next month or year. The rolled over quantity will be good for one additional month/year depending on the plan duration.
  • 60+ regions: Run your tests in 60+ regions across the globe and across the three major cloud providers (AWS, Azure, Google Cloud) or setup a self-hosted region anywhere in the world on your own infrastructure.

Add-Ons

Add-ons are additional products you can add to your account that are not directly included in any plan. Currently there are a few different possibilities with the specific pricing dependent on your plan:

Elastic IPs: If your systems require the IP addresses that client traffic originates from to be whitelisted and you want to use Testable hosted test runners, this add-on allows you to purchase elastic IPs in any AWS location. Once purchased if you drill down into the add-on product you can see the IP addresses assigned to your account. You can also see all tests that utilize those elastic IPs.

Elastic IP

Virtual User Hours: Purchase a block of virtual user hours and use them whenever you want before they expire (typically after 1 year).

Test Runner Hours: Purchase a block of test runner hours and use them whenever you want before they expire (typically after 1 year).

Billing Cycle

The monthly or annual billing cycle is based on the moment you sign up for the plan. So for example a monthly plan that was bought on April 18, 2022 will renew on the 18th of each month.

Plan Usage

Login to Testable then navigate to Org Management => Billing.

This page will show you an overview of your plans, usage, outstanding balance, allocations, etc. We try to give you lots of information to ensure you have the best set of plans purchased for your team.

Billing overview

The Products table gives you an idea of the usage this billing period as well as some historical stats to help you determine how well you’re utilizing your plans.

If you click into any product or plan you can see the usage history by billing period.

Billing Details

You can even drill down further to see every single test/monitor that was billed against that subscription.

Billing Usage

Limits and Alerts

Login to Testable then navigate to Org Management => Billing => Alerts to set an overall balance alert. You will receive an email alert if the balance exceeds that threshold.

Billing Alerts

You can also limit usage of any individual product via Org Management => Billing => Products => Limits column. These limits get checked both before starting a test and also as tests run. If a running test exceeds any limit it is stopped immediately.

Billing Limits

Settings

There are a few settings that affect how we utilize your plans when testa are run. These settings can be specified at both the organization level (via Org Management => Settings => Billing) or per test configuration (either as an API parameter, remote Selenium/Puppeteer/Playwright parameter, or below the test summary).

  1. Billing Strategy: Setting to help Testable decide when to run your test.
    • Minimize Cost: Run this test only when it can be done at the cheapest possible price or utilizing the fewest possible consumable units (e.g. test runner hours, virtual user hours, monitors, etc). The test will queue to run later if that is not possible now. For example: if you have a 2 browser sessions plan + a pay-as-you-go plan and there are already 2 browser sessions in progress Testable will not try to use the pay as you go plan to run your test and instead wait until one of the browser sessions is available for use.
    • ASAP: Run this test as soon as possible regardless of cost. If multiple plans are available right now to pay for your test the cheapest option will still be used. For example: if you have a 2 browser sessions plan + a pay-as-you-go plan and there are already 2 browser sessions in progress Testable will still immediately run your test using your pay as you go plan.
  2. Choosing Between Plans: When you want Testable to run your test ASAP, let us know whether to use the cheapest plan or if you prefer to explicitly choose the plan category (e.g. test runner vs virtual user vs browser automation session).

Share Across Organizations

You can share your plan across multiple organizations by linking them together. See the organizations page for more details.

What is a Test Runner?

This section is important if you have a plan where the billing is based on test runners. Test runners run your test and simulate virtual users as defined in your scenario. Test can be allocated across multiple test runners in one or more regions around the world. Read out test runners guide for more details.

Testable supports two general types of test runners: Per Test and Long Running.

Per Test: Test runner VMs will be spun up at the start of a test and automatically terminated at the end, either in your virtual network or ours. A t3.large (2 vCPU, 8GB RAM), or equivalently sized VM on Google Cloud and Azure, is considered 1 test runner. For larger or smaller instance types the test running billing equivalent is adjusted accordingly. For AWS this ranges from t3.nano = 1/16th of a test runner up to m5.8xlarge (32 vCPU, 128 GB RAM) = 16 test runner equivalents.

Long Running: When a test runs on a long running test runner we measure the test runner equivalent based on the peak number of vCPUs (1 minute moving average) and GB of RAM used during test execution. 1 test runner node corresponds to 2 vCPUs + 8GB RAM (measured to the nearest 0.25 test runners). On the test results you can view vCPU and RAM utilization in the last widget at the bottom (Test Runner Utilization).

Some examples:

  1. 6 vCPUs + 23GB RAM = 2 test runner equivalents
  2. 2 vCPUs + 1GB RAM = 0.5 test runner equivalents
  3. 16 vCPUs + 2GB RAM = 4 test runner equivalents
  4. 1.5 vCPU + 120GB RAM = 12 test runner equivalents