- Shared Test Runners
- On Demand Test Runners
- Test Configuration
Test runners run your test and simulate virtual users as defined in your scenario. Test can be allocated across multiple test runners in one or more regions around the world. The test runner agent can run on any machine.
There are two sources of test runners:
Shared: A shared grid of test runners can run one or more tests concurrently and can consistent of many machines. Shared grids can be private to your organization or public for sharing across all Testable customers.
On Demand: Testable can spin up a set of test runners specifically for one test only. Once initialized they will simulate all the virtual users defined in your test. Once finished Testable will terminate all test runners.
Shared grids execute your test immediately assuming there is capacity instead of waiting for infrastructure to be allocated.
Testable provides a public shared grid distributed across the globe on the infrustructure of various cloud providers. Tests can be run on this grid but keep in mind it is shared by all Testable users.
A private shared grid can also be setup that is shared across tests witin your organization only. This can be a useful way to leverage your own data centers and existing infrastructure for running load tests. It also allows you to use the same test runners across multiple tests and avoid the wait for on demand infrastructure on each test.
The My Shared Grid test runner source is automatically created the first time you register or run a private shared test runner.
There are three ways to start test runners in a private shared grid:
Run on your own infrastructure using Docker. Test runners are grouped by region name so that your team can have several different pools for different types of tests (e.g. load tests vs continuous integration). Whatever region names you provide when using the
docker run command will appear in the list.
Use the Testable Agent AMI available on AWS.
- Start EC2 instances using the latest version of the AMI called Testable Agent via the AWS Console or API. Alternatively, on Testable go to Organization => Test Runner Sources and press the Register Shared Instances button. A dropdown will appear near the top where you can select the desired AWS region and open the EC2 launch wizard with the right AMI automatically selected for you.
- Login to your Testable account and go to Organization => Test Runner Sources.
- Press the Register Shared Instances button.
- Enter a comma separated list of instance ids or public host names from step 1. Test runners will take a few minutes to appear here after launch.
- Choose the test runners to register as well as the desired region name. This is the region name that you can then choose in your test configuration (Test Runners => My Shared Grid => [region]). Tests will be evenly distributed across the test runners in the region.
View all registered test runners in a region by clicking its name.
Similar to option 2 above (Testable AWS AMI) except you pay per hour via the AWS marketplace and can run as many tests as you want as long as the tests run in a region with all test runners being unlimited.
On demand test runners are started by Testable at the start of a test run and are dedicated to your test only. Once the test finishes, these test runners are terminated.
Run your test on dedicated on-demand AWS EC2 instances, using either Testable's account or self-hosting within your own. Other cloud platforms will be added in the future.
Select AWS - Testable Account to have the EC2 test runner instances hosted within Testable's account.
Configure your own AWS account to self-host the EC2 test runner instances. Testable will automatically terminate the instances once the test is finished in that region. Click the New... option if you are setting up your AWS account for the first time and make sure to read our Self-Hosted AWS Test Runner Setup guide for detailed requirements.
Notice the extra settings:
- VPC: The AWS VPC where you want the test runners to run. Make sure the VPC chosen can connect the
- Subnet: The subnet within the chosen VPC where you want the test runners to run.
- Tags: Additional tags to add to any EC2 instance started. These can be specified either in the test configuration or a default set when creating/updating the test runner configuration.
- Key Pair:Choose a key pair you have access to so that you can connect and monitor the instances we spun up as part of your test.
When creating a new test configuration you must select one or more test runner sources to run your test. The same test runner source can be configured multiple times within a single configuration as long as you stay within your account limits.
So for example you could run your scenario on our shared public grid, inside your data center, and using on demand AWS EC2 instances all at the same time.
Go to Organization -> Test Runner Source to manage your sources.
Press the Add On Demand Source button in the upper left to add a new test runner. Currently the only on demand source is AWS.
Read the Self-Hosted AWS Test Runner Setup guide for step-by-step instructions.
Update your private AWS credentials, tags, and default VPC settings by clicking the configure icon next to the test runner source name.
Delete your AWS account as a source by clicking the trash icon next to the test runner source name.