AI Scenario

Introduction
How it Works
Defining Instructions
Instruction Templates
Configuration Options
Script Generation and Reuse
Running via API
Parameters

Introduction

The AI scenario type lets you describe your test in natural language and have Testable automatically generate and execute a Playwright browser automation script. No code required — just describe what you want each virtual user to do and Testable handles the rest.

This is useful for:

Quickly prototyping functional or monitoring tests without writing Playwright code
Non-engineers who want to define test steps in plain language
Smoke and sanity checks that need to be created or updated rapidly

How it Works

You provide one or more instructions — each with a title and a description of what the browser should do.
At test execution time, Testable sends your instructions to the selected AI model, which generates a Playwright script.
The generated script is executed on the test runner(s) just like any other Playwright scenario.
Optionally, the generated script is cached so subsequent runs reuse it without calling the AI again (configurable via Script Generation and Reuse).

Defining Instructions

Instructions are the core of an AI scenario. Each instruction has two fields:

Field	Description
Title	A short label for this step (e.g. “Login flow”)
Contents	A natural language description of what the browser should do (e.g. “Navigate to /login, enter the username `testuser` and password `secret`, click Submit, and verify the dashboard loads”)

You can add multiple instructions to compose a multi-step user journey. Instructions are executed in order and can be reordered by dragging them in the UI.

Tips for writing good instructions:

Be specific about the actions to take and the outcomes to verify (e.g. “verify that the page title contains ‘Dashboard’”)
Reference exact URLs, button labels, or field names where possible
Keep each instruction focused on a single logical step or flow to make failures easier to diagnose

Instruction Templates

Testable includes a library of instruction templates to help you get started quickly. Templates are available for common patterns:

Category	Templates
Authentication	Login Flow, Logout Flow, Signup Registration
Navigation	Page Navigation, Multi-Page Flow
Forms	Form Submission, Form Validation
E-Commerce	Add to Cart, Checkout Flow
Search & Filter	Search Functionality, Filter Results

Select Add Step in the Instructions tab and choose a template to insert a pre-filled instruction you can customize.

Configuration Options

Option	Description	Default
Model	The AI model used to generate the Playwright script. Valid model names can be retrieved from `GET /ai-models`. Set to Auto to use your organization’s default model.	Auto (org default)
Tool	The browser automation tool. Currently only `Playwright` is supported.	`Playwright`
Browser Type	Which browser Playwright launches. Options: `chromium`, `firefox`, `webkit`.	`chromium`
Reuse	Whether to cache the generated script for reuse (see below).	Reuse generated script
Generated script valid for	How long a cached script remains valid before the AI regenerates it. Specify a quantity and unit (minutes, hours, or days).	1 day
Regenerate script on failure	If the generated script fails at runtime, automatically ask the AI to regenerate it and retry. Only applicable when Reuse is enabled.	Enabled

Script Generation and Reuse

Calling the AI to generate a script on every test run adds latency and cost. Testable offers a Reuse option to cache the generated script:

Reuse generated script (default): The first execution generates the script. Subsequent runs use the cached script until the configured validity period expires, at which point the AI regenerates it.
Always run via AI: The AI generates a fresh script on every test run. Use this if your instructions change frequently or you always want the latest generation.

When Regenerate script on failure is enabled, a runtime failure triggers the AI to produce a new script and immediately retry the test. This helps recover from transient generation issues without manual intervention.

You can also download the most recently generated Playwright script directly from the scenario page using the Download Generated Script button. This is useful if you want to inspect what the AI produced or graduate the test to a manually maintained Playwright scenario.

Running via API

AI scenarios can be created and executed via the Simple Test API. Provide the ai_instructions parameter to automatically set the scenario type to AI.

Example:

curl \
  -F "scenario_type=AI" \
  -F 'ai_instructions=[{"title":"Login flow","contents":"Navigate to /login, enter username and password, click Submit, verify redirect to dashboard"}]' \
  -F "browser_type=chromium" \
  -F "concurrent_users=1" \
  -F "iterations=1" \
  -F "conf_testrunners[0].regions=aws-us-east-1" \
  -H "api_key: YOUR_API_KEY" \
  https://api.testable.io/start

AI-specific API parameters:

Parameter	Description	Required
`ai_instructions`	JSON array of instruction objects. Each object must have a `title` (string) and `contents` (string). Providing this parameter automatically sets the scenario type to AI. Example: `[{"title":"Login flow","contents":"Navigate to /login, enter credentials, verify dashboard loads"}]`	Yes
`model`	AI model name to use. Valid values from `GET /ai-models`. If not specified, uses the organization default.	No
`tool`	Browser automation tool. Currently only `Playwright` is supported.	No
`browser_type`	Browser to use: `chromium` (default), `firefox`, or `webkit`.	No
`max_age`	Milliseconds a generated script remains valid before regeneration. Defaults to `86400000` (1 day). Set to `0` to always regenerate.	No
`enable_ai_retry`	Whether to regenerate the script and retry if it fails at runtime. Defaults to `true`.	No

See the Simple Test API reference for the full list of common parameters including test runner, concurrency, duration, KPIs, and notifications.

Parameters

Like all Testable scenario types, AI scenarios support scenario parameters. You can reference parameter values from your instructions by name — the AI will incorporate them into the generated script as environment variables.

For example, define a parameter BASE_URL with value https://staging.myapp.com and reference it in your instructions: “Navigate to the ${BASE_URL} environment variable and verify the home page loads.”

Read more about scenario parameters here.