Run Tests and Read Reports

Executions are the concrete runs of a scenario + configuration pair. This page covers how to trigger runs, monitor them in real time, read the resulting reports, and customize what stakeholders see.

Kick off an execution

  • From the UI: open a configuration, pick the scenario version, and click Run Test. Watch live logs, video, metrics, and runner health on the same page.
  • Via schedules: set recurring runs (e.g., hourly, nightly, weekly) for continuous monitoring.
  • CI/CD & triggers: use trigger URLs, GitHub Action, GitLab, Jenkins, CircleCI, or webhook integrations from Continuous Integration & Triggers.
  • API: call the Execution API or the Simple Test API to start runs programmatically.

Execution lifecycle

  1. Queued: runners are allocated, artifacts fetched, parameters resolved.
  2. Provisioning: Testable spins up cloud or self-hosted workers and installs dependencies.
  3. Running: live metrics, logs, and artifacts stream to the dashboard; you can pause/resume/cancel if needed.
  4. Completed: results are locked, pass/fail determined, and alerts/integrations fire.
  5. Archived: artifacts remain downloadable, comparative trend charts update automatically.

Reading the report

  • Start with View Test Results for an orientation to the dashboard widgets.
  • Key panels include latency percentiles, throughput, error rates, resource usage, browser video/screenshot galleries, and any custom metrics you emit.
  • Use the timeline scrubber to inspect spikes, then drill into request traces, console logs, runner stats, and step-level screenshots.
  • Compare executions side by side to spot regressions or capacity shifts.

Customize dashboards & exports

  • Arrange or add widgets via Customize Test Results View.
  • Share read-only dashboards with teammates or stakeholders using secure links.
  • Export raw data (CSV/JSON), screenshots, HAR files, or custom metrics for offline analysis.
  • Push metrics to observability stacks like Datadog, New Relic, or CloudWatch through Integrations.

Alerts and success criteria

  • Attach Success Criteria & Breaking Points to fail executions automatically when SLAs drift.
  • Configure notification channels (Slack, Teams, email, SNS, webhooks) in the relevant integration page.
  • Use the API to pull execution summaries into internal dashboards or incident tooling.

Troubleshooting tips

  • Runner issues? Verify agent connectivity, credentials, or quotas via Test Runners.
  • Script failures? Use the per-run console logs, video replays, and trace files. For Playwright/Selenium/Puppeteer, enable verbose logging or traces in the scenario configuration.
  • Data drift? Inspect parameter values and uploaded datasets attached to the execution.

Next steps

  • Harden governance with Enterprise features like SSO, RBAC, audit logs, and private regions.
  • Automate remediation or create custom analytics using the API.
  • Connect downstream workflows through Integrations.