Testable On Premises

Introduction

The entire Testable platform can be installed on your own infrastructure with no dependency or connection back to the Testable cloud.

The platform is a collection of Docker containers that can be installed via Docker Compose or a Helm chart onto your Kubernetes cluster. The test runners are organized into regions and actually run the tests and generate the virtual users.

Architecture

Prerequisites

System Requirements

  • Testable Services & Storage: Any Linux flavor, 4 vCPU, 16GB RAM, 50GB storage should be sufficient to install all components of Testable system and 1 test runner.
  • Test Runner: Test runners require any Linux flavor, ~100MB RAM, and 5GB storage when not running any tests. Resource utilization per virtual user depends on the scenario type (e.g. OpenFin vs Webdriver.io vs JMeter vs Node.js). The test runner agent will attempt to utilize all resources accessible to it for generating virtual users.

Required Container Images

If your company has its own container registry you will need to pull in the following container images available on Docker Hub:

Testable

Required Dependencies (if not hosted separately)

Installation

We offer two installation options for Testable: Docker Compose and a Helm Chart.

If you are unsure which option to choose, go with Docker Compose to install Testable across 1 or more virtual machines without any additional setup.

Installation - Docker Compose

Follow these steps to install Testable via Docker Compose:

  1. Download our package (https://testable-onprem.s3.amazonaws.com/testable-onprem-latest.zip).
  2. Unzip it onto a Linux host with at least 4 vCPU, 16GB RAM, and 50GB storage.
  3. Run setup.sh or manually install Docker + Docker Compose. If you ran setup.sh, log out and back in to the shell.
  4. Configure your deployment via the onprem.env.template file. Read the configuration section below for more details or read the instructions in the file itself.
  5. Launch the system by running start.sh.
  6. Once all containers launch, open http://myhost (or https://myhost if SSL is enabled) in your browser and wait for the system to fully initialize. At this point it will prompt you to configure an administrator account.

Installation - Helm Chart (Kubernetes/OpenShift)

If you run your container based applications on Kubernetes or OpenShift you can use our Helm chart to install Testable on your cluster.

More details coming soon.

Post-Installation Setup

Configure Administrator Account

Once your deployment is fully initialized and running you will be prompted to configure an administrator account.

Initial Setup

Invite Users

To invite users, simply click the admin menu in the upper right, select Users, and press the Add User button.

Admin Menu

Create User

The user will receive an invite email with a link to set their password the first time.

Setup Shared Organizations

Different teams in your company may want to have separate shared workspaces to collaborate on their test cases. You can create as many shared organizations as you need via the admin menu => Organizations. Press the Add Organization button to create a new one. Once it is created you will be redirected to a page where you can add users to this new organization.

Run Your First Test

To run your first test simply press the New Test Wizard button on the website and follow the steps. Getting Started guides for each tool are available on our documentation site.

Configuration

The Testable environment is configured via the onprem.env.template file provided in our installation package. This section explains each property and which ones are mandatory.

Required

  • TESTABLE_LICENSE: License key provided by the Testable team. If you do not have one please contact us at sales@testable.io to inquire.
  • TESTABLE_WEBSITE_URL: The URL by which you will access the Testable web application. If you've run Testable on a single host this will be the hostname or DNS alias you have setup to the host. If you've run Testable across multiple hosts this could be a DNS alias to a load balancer.
  • Email Settings: SMTP settings so that Testable can send email notifications.
    • TESTABLE_SMTP_HOST: The hostname for your SMTP provider
    • TESTABLE_SMTP_PORT: Port for accessing your SMTP provider (e.g. 587)
    • TESTABLE_SMTP_TLS: Set to true if accessing SMTP via a secured TLS connection
    • TESTABLE_SMTP_USERNAME: If your SMTP provider requires authentication, this is the username
    • TESTABLE_SMTP_PASSWORD: If your SMTP provider requires authentication, this is the password
    • TESTABLE_SMTP_NOTIFICATIONS_FROM: The email address to show in the FROM line of the email (e.g. Acme Support <support@acme.com>)
    • TESTABLE_SMTP_NOTIFICATIONS_TO: For system alerts, an email address to send the alerts to (e.g. support@acme.com)

Optional

  • SSL settings: When you unzip the Testable package there will be an empty ssl directory created. Any SSL related files should be placed in this directory since it is mounted as a volume on the test runner (/ssh). The following settings are for enabling SSL on all Testable services and the website. See the onprem.env.template file for more details. Make sure your paths start with /ssl.
  • SAML settings: To enable SAML authentication for your Testable installation. See the onprem.env.template file for more details.
  • AWS Cloud settings: Testable can spin up on demand test runners in your AWS VPC if given access. See the onprem.env.template file for more details. This will not result in any connections back to Testable cloud.
  • Azure Cloud settings: Testable can spin up on demand test runners in your Azure virtual network if given access. See the onprem.env.template file for more details. This will not result in any connections back to Testable cloud.
  • Recording proxies: Settings related to setting up an HTTP forward proxy and an HTTP reverse proxy that are able to record traffic for replaying as a test scenario. See the onprem.env.template file for more details.

Using Your Own Kafka, Zookeeper, Redis Cluster, MySQL

By default our deployment package includes a self-bundled Kafka cluster, Zookeeper cluster, Redis cluster, and a MySQL database.

If you have these running inside your organization as a shared service you can decide to not include these containers and instead point the Testable installation at your existing services. This is done via the following settings in your onprem.env.template file:

  • TESTABLE_KAFKA_BROKERS: Comma separated list of Kafka brokers
  • TESTABLE_ZOOKEEPER_SERVERS: Comma separated list of Zookeeper servers
  • TESTABLE_REDIS_NODES: Comma separated list of Redis cluster nodes. Note that Testable only works with Redis cluster, not with a single Redis master/slave deployment.
  • MySQL Databases: Testable creates and initializes two databases when initializing for the first time: testable and testable-results. If you want to use an existing MySQL host, make sure both of these databases are created or the mysql user has permissions to create new databases. These two databases can be on different hosts.
    • For the testable database: TESTABLE_DB_MAIN_USERNAME, TESTABLE_DB_MAIN_PASSWORD, TESTABLE_DB_MAIN_HOST, TESTABLE_DB_MAIN_PORT
    • For the testable-results database: TESTABLE_DB_RESULTS_USERNAME, TESTABLE_DB_RESULTS_PASSWORD, TESTABLE_DB_RESULTS_HOST, TESTABLE_DB_RESULTS_PORT

Updates

For Docker Compose installations simply run the update.sh script to pull in new versions of container images. Testable will automatically handle applying updates across all components once a new version of the container images have been pulled.

For Helm, more instructions coming soon.

Launching More Test Runners

Scaling the default region

To launch additional test runners in the default region simply run docker-compose scale test-runners=COUNT where COUNT is the number of test runners you want to spin up across your Docker Swarm.

Creating new test runner regions

Test runner containers can be run on any instance that is able to connect to your central Testable deployment. If launched within the same Docker network as the core services you only need to specify the region name and it will automatically find the test runner coordinator service to register with.

Outside of the Docker network you will need to specify the server URL for the test runner to connect to (AGENT_SERVER_URL) which is either:

  • The website url with the path /agents. So http://my-website-url/agents.
  • The path to the test runner coordinator service. By default it is exposed on port 8080 of the hosts on which you deployed your Testable environment (e.g. http://my-website-url:8080).

For example, with Docker compose:

my-test-runner:
    image: testable/agent:latest
    env_file:
      - onprem.env
    environment:
      - AGENT_REGION_NAME=my-region
      - AGENT_REGION_SCOPE=public
      - AGENT_SERVER_URL=http://my-website-url/agents
    security_opt:
      - seccomp:unconfined
    privileged: true

If you want this test runner region to be available to all users of your Testable installation make sure to pass AGENT_REGION_SCOPE=public.

If you prefer it only be available to a particular organization, then do the following:

  1. Login to your Testable web (http://my-website-url)
  2. Switch to the Organization you want to target via the Organizations dropdown at the top
  3. Go to Org Management => API Keys to create a new key
  4. Pass this to your test runners using AGENT_KEY=xxxx and make sure to omit the AGENT_REGION_SCOPE variable.

Administration

Administrators can see a special menu in the upper right corner that allows them to administer the platform.

Admin Menu

See all running tests & test runners

Select Running Tests & Runners via the admin menu to see all tests currents running, the total traffic those tests are generating, and how details of all currently active test runners.

Running Tests & Runners

Manage users

Select Users via the admin menu to add new users, delete existing users, and toggle on/off super administrator permissions.

Manage Users

Manage organizations

Select Organizations via the admin menu to add new shared organizations, delete existing organizations, and manage the users in each organization.

Manage Organizations