AI Conversations

Introduction

AI Conversations provide an agent workspace inside Testable. Use it to ask questions, investigate executions, summarize account health, and take action across the same testing resources you manage in the web app.

In Agent mode, AI Conversations are designed to support the same kinds of actions you can perform in the Testable web app, including creating and updating test cases, scenarios, configurations, executions, schedules, success criteria, notifications, integrations, runners, organization settings, and related testing resources.

Open AI Conversations

Open the AI Conversations page from the Testable app sidebar, or navigate directly to:

https://a.testable.io/conversations

The page opens a full chat workspace with a message composer, model selector, runner region selector, mode selector, file upload action, settings button, and conversation history panel.

What the Agent Can Do

The agent understands Testable concepts and can work across the full testing workspace. The examples below are starting points, not a complete list of supported actions.

  • Summarize tests and executions - Ask for status, trends, and areas of concern across recent runs.
  • Create or update scenarios - Describe browser journeys, API checks, load scripts, or AI-generated tests in plain language.
  • Create configurations - Ask the agent to set load, duration, regions, browsers, schedules, success criteria, parameters, and data files.
  • Start executions - Ask the agent to run a configuration and explain the live or completed results.
  • Investigate failures - Ask it to review logs, screenshots, traces, metrics, errors, and reports.
  • Manage integrations and notifications - Ask who receives alerts or update notification and integration settings.
  • Update organization settings - Ask for help with shared defaults and testing configuration normally managed in the web app.

Available actions depend on your authenticated user permissions, plan, feature flags, and rollout state. If your user can perform an action in the web app, an AI Conversation can generally perform that action through approved tools.

Conversation Modes

  • Agent - Can make changes to your testing resources when tool calls are approved.
  • Ask - Read-only mode for questions, summaries, and explanations.

Use Ask mode when you only want analysis. Switch to Agent mode when you want the assistant to prepare or apply changes.

Tool Approval

When the agent needs to call a tool, Testable shows the tool call in the conversation. Approve individual calls when you want the action to run.

For repeated trusted actions, use Always Approve. Auto-approved tools appear in the settings modal under Auto-Approved Tools and can be removed later.

Review tool calls before approval, especially when they create, update, start, stop, archive, or delete testing resources.

History and Attachments

Conversations are saved in the right-side panel. You can create a new agent, reopen previous conversations, rename conversations, and delete conversations. Conversation URLs include a conversation ID so a specific thread can be reopened directly.

Use the upload action to attach files to the active conversation. When an agent uses browser-based tools, the conversation header can show browser session activity and available artifacts. The conversation also tracks metadata such as elapsed time, model, cost, and context usage when available.

AI and MCP Settings

Open the settings icon in the upper right of the conversations panel to configure AI and MCP settings. AI settings include using your own OpenAI key, optionally setting a custom OpenAI-compatible base URL, and defining an organization-level system message.

The settings modal also provides the Testable MCP server configuration. See Testable MCP Server for details.