Route Description
GET /test-confs/:confId/executions Get list of executions
GET /executions/:id Get execution details
GET /executions/:id/summary Get execution summary
GET /executions/:id/metric Get execution metric
GET /executions/:id/results[.csv] Download execution results
GET /executions/:id/suite-results[.csv] Download assertion results
GET /executions/:id/suite-summaries[.csv] Download assertion summary
GET /executions/:id/suite-errors[.csv] Download assertion error summary
GET /executions/:id/traces.zip Download traces
GET /executions/:id/output-files Get output file metadata
GET /executions/:id/output-files.zip Download zip file of all output files
GET /executions/:id/output-files/:fileName/download Download file by name or id
GET /executions/:id/results.pdf Export to PDF
GET /executions/:id/suite-results[.csv] Download assertion results
GET /executions/:id/intervals Get execution intervals
GET /executions/:id/resources Get execution results by resource
GET /executions/:id/resources/:resourceId Get execution results for a resource
GET /executions/:id/logs[.txt] Get execution logs
GET /executions/:id/traces Get execution traces
GET /execution-traces/:id Get execution trace
POST /test-confs/:confId/executions Start execution
HEAD /executions/:id Check if execution exists
PUT /executions/:id/manual-start Manual start
PUT /executions/:id/stop Stop execution
DELETE /executions/:id Delete execution
POST /executions/:id/add-users Add users/engines
POST /executions/:id/live-extension Extend execution
POST /executions/:id/send-event Send event
POST /orgs/:orgId/tokens Share results
POST /executions/:id/tags Tag results
POST /search Search for results
POST /executions/:id/evaluate-kpis Evaluate Test Result

Overview

API routes related to test executions. A test execution is a load test that runs a scenario under a certain configuration. Each execution contains one or more regions where it executes. Within each execution region, the work is split up into “chunks”. Test execution results are aggregated in 10 second intervals. Result metrics are grouped by resource.

Get list of executions

Retrieve a list of all executions within a given test configuration.

GET /test-confs/:confId/executions

Request Parameters

  • down: Whether or not to include execution region/chunk information. Defaults to true.
  • up: Whether or not to include the parent details. Defaults to false.
  • includeScript: Whether or not to include the script information used to run the test. Defaults to true.
  • includeSummary: Whether or not to include the metric summary. Defaults to true.
  • includeResources: Whether or not to include the by resource results. Defaults to false.
  • offset: Offset within the executions to start the response. Defaults to 0. When combined with limit it is useful for paging through the results.
  • period: What time period to query past executions in. Possible values are recent (last 15 or limit parameter value), 7d (last 7 days), 1m (last 1 month), 3m (last 3 months). Defaults to recent if start/end not specified.
  • start: The start createdAt timestamp (Unix epoch in milliseconds, UTC). If specified, the period parameter is ignored.
  • end: The end createdAt timestamp (Unix epoch in milliseconds, UTC). Defaults to the current timestamp if start is specified.
  • limit: The number of results to return. Defaults to 15 unless a period other than recent is specified in which case it defaults to -1 (i.e. return entire history).

Sample Response

[
   {
      "id":637,
      "createdAt":"2016-02-18T19:11:52.737Z",
      "updatedAt":"2016-02-18T19:15:26.544Z",
      "running":false,
      "allocated":true,
      "startedAt":"2016-02-18T19:11:54.095Z",
      "finishedAt":"2016-02-18T19:15:25.150Z",
      "cancelledByUser":false,
      "completed":true,
      "summaryCalculated":true,
      "limitBreached":false,
      "iterations":20,
      "iterationSleepSecs":10,
      "rampupSecs":0,
      "regions":[
         {
            "id":669,
            "createdAt":"2016-02-18T19:11:52.744Z",
            "updatedAt":"2016-02-18T19:15:25.130Z",
            "running":false,
            "startedAt":"2016-02-18T19:12:06.045Z",
            "finishedAt":"2016-02-18T19:15:25.130Z",
            "completed":true,
            "concurrentClients":5,
            "actualConcurrentClients":5,
            "expectedResultCount":100,
            "region":{
               "id":1,
               "createdAt":"2015-08-11T17:03:34.761Z",
               "updatedAt":"2015-08-11T17:03:34.761Z",
               "name":"aws-us-east-1",
               "public":true,
               "latitude":39.0436,
               "longitude":-77.4878,
               "description":"AWS N. Virginia",
               "active":true
            },
            "summary":{
               "id":206865,
               "createdAt":"2016-02-18T19:15:25.749Z",
               "updatedAt":"2016-02-18T19:15:25.749Z",
               "executionId":637,
               "executionRegionId":669,
               "intervalStart":0,
               "iterationsExecuted":100,
               "count":100,
               "concurrentClients":5,
               "actualConcurrentClients":0,
               "maxConcurrentClients":0,
               "metrics":[
                  {
                     "metricDef":"httpResponseCode",
                     "metricType":"Histogram",
                     "metricValueMap":{
                        "200":100
                     }
                  },
                  {
                     "metricDef":"connectionOpenMs",
                     "metricType":"Timing",
                     "metricValueMap":{
                        "count":100,
                        "mean":5,
                        "min":1,
                        "p95":9,
                        "max":214,
                        "p99":41,
                        "sd":21,
                        "sum":527,
                        "var":481
                     },
                     "metricUnits":"ms"
                  },
                  {
                     "metricDef":"bandwidth",
                     "metricType":"Counter",
                     "metricValue":17261,
                     "metricUnits":"bytes"
                  },
                  {
                     "metricDef":"success",
                     "metricType":"Counter",
                     "metricValue":100,
                     "metricUnits":"requests"
                  },
                  {
                     "metricDef":"dataPacketsSent",
                     "metricType":"Counter",
                     "metricValue":100,
                     "metricUnits":"packets"
                  },
                  {
                     "metricDef":"connectionCloseMs",
                     "metricType":"Timing",
                     "metricValueMap":{
                        "count":100,
                        "mean":8,
                        "min":3,
                        "p95":12,
                        "max":221,
                        "p99":43,
                        "sd":22,
                        "sum":886,
                        "var":497
                     },
                     "metricUnits":"ms"
                  },
                  {
                     "metricDef":"bytesReceived",
                     "metricType":"Counter",
                     "metricValue":9961,
                     "metricUnits":"bytes"
                  },
                  {
                     "metricDef":"dataPacketsReceived",
                     "metricType":"Counter",
                     "metricValue":200,
                     "metricUnits":"packets"
                  },
                  {
                     "metricDef":"firstReceivedMs",
                     "metricType":"Timing",
                     "metricValueMap":{
                        "count":100,
                        "mean":8,
                        "min":3,
                        "p95":11,
                        "max":217,
                        "p99":43,
                        "sd":21,
                        "sum":847,
                        "var":482
                     },
                     "metricUnits":"ms"
                  },
                  {
                     "metricDef":"bytesSent",
                     "metricType":"Counter",
                     "metricValue":7300,
                     "metricUnits":"bytes"
                  },
                  {
                     "metricDef":"dnsLookupMs",
                     "metricType":"Timing",
                     "metricValueMap":{
                        "count":100,
                        "mean":4,
                        "min":0,
                        "p95":8,
                        "max":212,
                        "p99":40,
                        "sd":21,
                        "sum":406,
                        "var":477
                     },
                     "metricUnits":"ms"
                  }
               ]
            },
            "chunks":[
               {
                  "id":638,
                  "executionType":"Main",
                  "agent":"7316fdd5-a620-401f-ae3a-9c9468489ed4",
                  "createdAt":"2016-02-18T19:12:06.082Z",
                  "updatedAt":"2016-02-18T19:15:25.111Z",
                  "running":false,
                  "startedAt":"2016-02-18T19:12:06.499Z",
                  "finishedAt":"2016-02-18T19:15:25.111Z",
                  "completed":true,
                  "iterations":20,
                  "iterationSleepSecs":10,
                  "rampupSecs":0,
                  "concurrentClients":5,
                  "actualConcurrentClients":5,
                  "completedDurationSecs":0,
                  "completedIterations":0
               }
            ]
         }
      ],
      "concurrentClients":5,
      "actualConcurrentClients":5,
      "expectedResultCount":100,
      "scriptParams":"{}",
      "summary":{
         "id":206866,
         "createdAt":"2016-02-18T19:15:25.749Z",
         "updatedAt":"2016-02-18T19:15:25.749Z",
         "executionId":637,
         "intervalStart":0,
         "iterationsExecuted":100,
         "count":100,
         "concurrentClients":5,
         "actualConcurrentClients":0,
         "maxConcurrentClients":0,
         "metrics":[
            {
               "metricDef":"httpResponseCode",
               "metricType":"Histogram",
               "metricValueMap":{
                  "200":100
               }
            },
            {
               "metricDef":"connectionOpenMs",
               "metricType":"Timing",
               "metricValueMap":{
                  "count":100,
                  "mean":5,
                  "min":1,
                  "p95":9,
                  "max":214,
                  "p99":41,
                  "sd":21,
                  "sum":527,
                  "var":481
               },
               "metricUnits":"ms"
            },
            {
               "metricDef":"bandwidth",
               "metricType":"Counter",
               "metricValue":17261,
               "metricUnits":"bytes"
            },
            {
               "metricDef":"success",
               "metricType":"Counter",
               "metricValue":100,
               "metricUnits":"requests"
            },
            {
               "metricDef":"dataPacketsSent",
               "metricType":"Counter",
               "metricValue":100,
               "metricUnits":"packets"
            },
            {
               "metricDef":"connectionCloseMs",
               "metricType":"Timing",
               "metricValueMap":{
                  "count":100,
                  "mean":8,
                  "min":3,
                  "p95":12,
                  "max":221,
                  "p99":43,
                  "sd":22,
                  "sum":886,
                  "var":497
               },
               "metricUnits":"ms"
            },
            {
               "metricDef":"bytesReceived",
               "metricType":"Counter",
               "metricValue":9961,
               "metricUnits":"bytes"
            },
            {
               "metricDef":"dataPacketsReceived",
               "metricType":"Counter",
               "metricValue":200,
               "metricUnits":"packets"
            },
            {
               "metricDef":"firstReceivedMs",
               "metricType":"Timing",
               "metricValueMap":{
                  "count":100,
                  "mean":8,
                  "min":3,
                  "p95":11,
                  "max":217,
                  "p99":43,
                  "sd":21,
                  "sum":847,
                  "var":482
               },
               "metricUnits":"ms"
            },
            {
               "metricDef":"bytesSent",
               "metricType":"Counter",
               "metricValue":7300,
               "metricUnits":"bytes"
            },
            {
               "metricDef":"dnsLookupMs",
               "metricType":"Timing",
               "metricValueMap":{
                  "count":100,
                  "mean":4,
                  "min":0,
                  "p95":8,
                  "max":212,
                  "p99":40,
                  "sd":21,
                  "sum":406,
                  "var":477
               },
               "metricUnits":"ms"
            }
         ]
      },
      "script":{
         "id":273,
         "createdAt":"2016-02-18T19:00:46.862Z",
         "updatedAt":"2016-03-09T08:10:44.945Z",
         "name":"Default Scenario",
         "length":97,
         "hasInit":false,
         "hasTeardown":false,
         "scriptType":"Code"
      }
   },
   ...
]

Get execution details

Retrieve details of a single execution. If the completed field is not true, then the test is still running.

GET /executions/:id

Request Parameters

  • down: Whether or not to include execution region/chunk information. Defaults to true.
  • up: Whether or not to include the parent details. Defaults to true.
  • includeScript: Whether or not to include the script information used to run the test. Defaults to true.
  • includeSummary: Whether or not to include the metric summary. Defaults to true.
  • includeIntervals: Whether or not to include the by interval results. Defaults to false.
  • includeResources: Whether or not to include the by resource results. Defaults to false.

Sample Response

{
   "id":637,
   "createdAt":"2016-02-18T19:11:52.737Z",
   "updatedAt":"2016-02-18T19:15:26.544Z",
   "running":false,
   "allocated":true,
   "startedAt":"2016-02-18T19:11:54.095Z",
   "finishedAt":"2016-02-18T19:15:25.150Z",
   "cancelledByUser":false,
   "completed":true,
   "summaryCalculated":true,
   "limitBreached":false,
   "iterations":20,
   "iterationSleepSecs":10,
   "rampupSecs":0,
   "regions":[
      {
         "id":669,
         "createdAt":"2016-02-18T19:11:52.744Z",
         "updatedAt":"2016-02-18T19:15:25.130Z",
         "running":false,
         "startedAt":"2016-02-18T19:12:06.045Z",
         "finishedAt":"2016-02-18T19:15:25.130Z",
         "completed":true,
         "concurrentClients":5,
         "actualConcurrentClients":5,
         "expectedResultCount":100,
         "region":{
            "id":1,
            "createdAt":"2015-08-11T17:03:34.761Z",
            "updatedAt":"2015-08-11T17:03:34.761Z",
            "name":"aws-us-east-1",
            "public":true,
            "latitude":39.0436,
            "longitude":-77.4878,
            "description":"AWS N. Virginia",
            "active":true
         },
         "summary":{
            "id":206865,
            "createdAt":"2016-02-18T19:15:25.749Z",
            "updatedAt":"2016-02-18T19:15:25.749Z",
            "executionId":637,
            "executionRegionId":669,
            "intervalStart":0,
            "iterationsExecuted":100,
            "count":100,
            "concurrentClients":5,
            "actualConcurrentClients":0,
            "maxConcurrentClients":0,
            "metrics":[
               {
                  "metricDef":"httpResponseCode",
                  "metricType":"Histogram",
                  "metricValueMap":{
                     "200":100
                  }
               },
               {
                  "metricDef":"connectionOpenMs",
                  "metricType":"Timing",
                  "metricValueMap":{
                     "count":100,
                     "mean":5,
                     "min":1,
                     "p95":9,
                     "max":214,
                     "p99":41,
                     "sd":21,
                     "sum":527,
                     "var":481
                  },
                  "metricUnits":"ms"
               },
               {
                  "metricDef":"bandwidth",
                  "metricType":"Counter",
                  "metricValue":17261,
                  "metricUnits":"bytes"
               },
               {
                  "metricDef":"success",
                  "metricType":"Counter",
                  "metricValue":100,
                  "metricUnits":"requests"
               },
               {
                  "metricDef":"dataPacketsSent",
                  "metricType":"Counter",
                  "metricValue":100,
                  "metricUnits":"packets"
               },
               {
                  "metricDef":"connectionCloseMs",
                  "metricType":"Timing",
                  "metricValueMap":{
                     "count":100,
                     "mean":8,
                     "min":3,
                     "p95":12,
                     "max":221,
                     "p99":43,
                     "sd":22,
                     "sum":886,
                     "var":497
                  },
                  "metricUnits":"ms"
               },
               {
                  "metricDef":"bytesReceived",
                  "metricType":"Counter",
                  "metricValue":9961,
                  "metricUnits":"bytes"
               },
               {
                  "metricDef":"dataPacketsReceived",
                  "metricType":"Counter",
                  "metricValue":200,
                  "metricUnits":"packets"
               },
               {
                  "metricDef":"firstReceivedMs",
                  "metricType":"Timing",
                  "metricValueMap":{
                     "count":100,
                     "mean":8,
                     "min":3,
                     "p95":11,
                     "max":217,
                     "p99":43,
                     "sd":21,
                     "sum":847,
                     "var":482
                  },
                  "metricUnits":"ms"
               },
               {
                  "metricDef":"bytesSent",
                  "metricType":"Counter",
                  "metricValue":7300,
                  "metricUnits":"bytes"
               },
               {
                  "metricDef":"dnsLookupMs",
                  "metricType":"Timing",
                  "metricValueMap":{
                     "count":100,
                     "mean":4,
                     "min":0,
                     "p95":8,
                     "max":212,
                     "p99":40,
                     "sd":21,
                     "sum":406,
                     "var":477
                  },
                  "metricUnits":"ms"
               }
            ]
         },
         "chunks":[
            {
               "id":638,
               "executionType":"Main",
               "agent":"7316fdd5-a620-401f-ae3a-9c9468489ed4",
               "createdAt":"2016-02-18T19:12:06.082Z",
               "updatedAt":"2016-02-18T19:15:25.111Z",
               "running":false,
               "startedAt":"2016-02-18T19:12:06.499Z",
               "finishedAt":"2016-02-18T19:15:25.111Z",
               "completed":true,
               "iterations":20,
               "iterationSleepSecs":10,
               "rampupSecs":0,
               "concurrentClients":5,
               "actualConcurrentClients":5,
               "completedDurationSecs":0,
               "completedIterations":0
            }
         ]
      }
   ],
   "concurrentClients":5,
   "actualConcurrentClients":5,
   "expectedResultCount":100,
   "scriptParams":"{}",
   "summary":{
      "id":206866,
      "createdAt":"2016-02-18T19:15:25.749Z",
      "updatedAt":"2016-02-18T19:15:25.749Z",
      "executionId":637,
      "intervalStart":0,
      "iterationsExecuted":100,
      "count":100,
      "concurrentClients":5,
      "actualConcurrentClients":0,
      "maxConcurrentClients":0,
      "metrics":[
         {
            "metricDef":"httpResponseCode",
            "metricType":"Histogram",
            "metricValueMap":{
               "200":100
            }
         },
         {
            "metricDef":"connectionOpenMs",
            "metricType":"Timing",
            "metricValueMap":{
               "count":100,
               "mean":5,
               "min":1,
               "p95":9,
               "max":214,
               "p99":41,
               "sd":21,
               "sum":527,
               "var":481
            },
            "metricUnits":"ms"
         },
         {
            "metricDef":"bandwidth",
            "metricType":"Counter",
            "metricValue":17261,
            "metricUnits":"bytes"
         },
         {
            "metricDef":"success",
            "metricType":"Counter",
            "metricValue":100,
            "metricUnits":"requests"
         },
         {
            "metricDef":"dataPacketsSent",
            "metricType":"Counter",
            "metricValue":100,
            "metricUnits":"packets"
         },
         {
            "metricDef":"connectionCloseMs",
            "metricType":"Timing",
            "metricValueMap":{
               "count":100,
               "mean":8,
               "min":3,
               "p95":12,
               "max":221,
               "p99":43,
               "sd":22,
               "sum":886,
               "var":497
            },
            "metricUnits":"ms"
         },
         {
            "metricDef":"bytesReceived",
            "metricType":"Counter",
            "metricValue":9961,
            "metricUnits":"bytes"
         },
         {
            "metricDef":"dataPacketsReceived",
            "metricType":"Counter",
            "metricValue":200,
            "metricUnits":"packets"
         },
         {
            "metricDef":"firstReceivedMs",
            "metricType":"Timing",
            "metricValueMap":{
               "count":100,
               "mean":8,
               "min":3,
               "p95":11,
               "max":217,
               "p99":43,
               "sd":21,
               "sum":847,
               "var":482
            },
            "metricUnits":"ms"
         },
         {
            "metricDef":"bytesSent",
            "metricType":"Counter",
            "metricValue":7300,
            "metricUnits":"bytes"
         },
         {
            "metricDef":"dnsLookupMs",
            "metricType":"Timing",
            "metricValueMap":{
               "count":100,
               "mean":4,
               "min":0,
               "p95":8,
               "max":212,
               "p99":40,
               "sd":21,
               "sum":406,
               "var":477
            },
            "metricUnits":"ms"
         }
      ]
   },
   "script":{
      "id":273,
      "createdAt":"2016-02-18T19:00:46.862Z",
      "updatedAt":"2016-03-09T08:10:44.945Z",
      "name":"Default Scenario",
      "length":97,
      "hasInit":false,
      "hasTeardown":false,
      "scriptType":"Code"
   }
}

Get execution summary

Returns summary metrics for this test execution.

GET /executions/:id/summary

Sample Response

{
   "id":206866,
   "createdAt":"2016-02-18T19:15:25.749Z",
   "updatedAt":"2016-02-18T19:15:25.749Z",
   "executionId":637,
   "intervalStart":0,
   "iterationsExecuted":100,
   "count":100,
   "concurrentClients":5,
   "actualConcurrentClients":0,
   "maxConcurrentClients":0,
   "metrics":[
      {
         "metricDef":"httpResponseCode",
         "metricType":"Histogram",
         "metricValueMap":{
            "200":100
         }
      },
      {
         "metricDef":"connectionOpenMs",
         "metricType":"Timing",
         "metricValueMap":{
            "count":100,
            "mean":5,
            "min":1,
            "p95":9,
            "max":214,
            "p99":41,
            "sd":21,
            "sum":527,
            "var":481
         },
         "metricUnits":"ms"
      },
      {
         "metricDef":"bandwidth",
         "metricType":"Counter",
         "metricValue":17261,
         "metricUnits":"bytes"
      },
      {
         "metricDef":"success",
         "metricType":"Counter",
         "metricValue":100,
         "metricUnits":"requests"
      },
      {
         "metricDef":"dataPacketsSent",
         "metricType":"Counter",
         "metricValue":100,
         "metricUnits":"packets"
      },
      {
         "metricDef":"connectionCloseMs",
         "metricType":"Timing",
         "metricValueMap":{
            "count":100,
            "mean":8,
            "min":3,
            "p95":12,
            "max":221,
            "p99":43,
            "sd":22,
            "sum":886,
            "var":497
         },
         "metricUnits":"ms"
      },
      {
         "metricDef":"bytesReceived",
         "metricType":"Counter",
         "metricValue":9961,
         "metricUnits":"bytes"
      },
      {
         "metricDef":"dataPacketsReceived",
         "metricType":"Counter",
         "metricValue":200,
         "metricUnits":"packets"
      },
      {
         "metricDef":"firstReceivedMs",
         "metricType":"Timing",
         "metricValueMap":{
            "count":100,
            "mean":8,
            "min":3,
            "p95":11,
            "max":217,
            "p99":43,
            "sd":21,
            "sum":847,
            "var":482
         },
         "metricUnits":"ms"
      },
      {
         "metricDef":"bytesSent",
         "metricType":"Counter",
         "metricValue":7300,
         "metricUnits":"bytes"
      },
      {
         "metricDef":"dnsLookupMs",
         "metricType":"Timing",
         "metricValueMap":{
            "count":100,
            "mean":4,
            "min":0,
            "p95":8,
            "max":212,
            "p99":40,
            "sd":21,
            "sum":406,
            "var":477
         },
         "metricUnits":"ms"
      }
   ]
}

Get execution metric

Returns aggregated metric details for a single metric. Works both during live test execution as well as after the test is finished running.

GET /executions/:id/metric?ns=[namespace]&name=[metric_name]

Sample Response

{
   "namespace": "Testable",
   "metricDef":"firstReceivedMs",
   "metricType":"Timing",
   "metricValueMap":{
      "count":100,
      "mean":8,
      "min":3,
      "p95":11,
      "max":217,
      "p99":43,
      "sd":21,
      "sum":847,
      "var":482
   },
   "metricUnits":"ms"
}

Download execution results

Download the raw results, unaggregated for this execution. Can also be downloaded for one region where the test executed or for one resource. Note that the regionId can be found in the execution details at regions[i].id. Omit the .csv at the end to get the results in JSON format.

GET /executions/:execution/results[.csv]
GET /executions/:execution/regions/:regionId/results[.csv]
GET /executions/:execution/resources/:resource/results[.csv]
GET /executions/:execution/regions/:regionId/resources/:resource/results[.csv]

Download assertion results

Download all assertions captured during this test. Includes any error messages and enough information to correlate the assertion with a particular virtual user, iteration, etc. Can also be downloaded for one region where the test executed or overall, either all assertions or just the ones with a failure. Note that the regionId can be found in the execution details at regions[i].id. Omit the .csv at the end to get the results in JSON format.

GET /executions/:execution/suite-results[.csv]
GET /executions/:execution/regions/:regionId/suite-results[.csv]
GET /executions/:execution/failed-suite-results[.csv]
GET /executions/:execution/regions/:regionId/failed-suite-results[.csv]

Request Parameters

  • limit: Number of assertions to return. Returns all by default or if set to -1.
  • offset: Offset within the result set to start the response. Defaults to 0. When combined with limit it is useful for paging through the assertion results.
  • threadId: Optional if you want to return results only for one user, pass the thread id.

Download assertion summary

Download a summary of all assertions captured during this test. Includes various counts like passed assertions, failed assertions, screenshots captured, etc. Can also be downloaded for one region where the test executed or overall. Note that the regionId can be found in the execution details at regions[i].id. Omit the .csv at the end to get the results in JSON format.

GET /executions/:execution/suite-summaries[.csv]
GET /executions/:execution/regions/:regionId/suite-summaries[.csv]

Request Parameters

  • includeErrors: Whether to include a row for each error encountered by assertion.

Download assertion errors summary

Download a summary of all errors that occurred while validating assertions in your test scenario. Can also be downloaded for one region where the test executed or overall. Note that the regionId can be found in the execution details at regions[i].id. Omit the .csv at the end to get the results in JSON format.

GET /executions/:execution/suite-errors[.csv]
GET /executions/:execution/regions/:regionId/suite-errors[.csv]

Download traces

Download a ZIP file containing details on all traces captured during this test. For each trace there will be one file that contains the trace JSON and a second file that contains the packets captured for that trace. The file names correspond to the resource name of the trace with spaces and special characters replaced.

GET /executions/:execution/traces.zip
GET /executions/:execution/regions/:regionId/traces.zip
GET /executions/:execution/resources/:resource/traces.zip
GET /executions/:execution/regions/:regionId/resources/:resource/traces.zip

Get output file metadata

Download information including file name, size, concurrent user #, iteration #, timestamp, etc for all files captured during a test run.

GET /executions/:execution/output-files

Request Parameters

  • limit: Number of output files to return. Returns all by default or if set to -1.
  • offset: Offset within the result set to start the response. Defaults to 0. When combined with limit it is useful for paging through the results.
  • since: Only return files where capturedAt >= since. Time is specified as a unix epoch in milliseconds (e.g. 1457636340000). Optional.
  • until: Only return files where capturedAt < until. Time is specified as a unix epoch in milliseconds (e.g. 1457636340000). Optional.
  • contentType: Only return files where the content type matches the specified value. Optional.
  • agent: Comma separated list of agent (i.e. test runner) ids to match. Only return files captured from the listed agents if specified. Optional.
  • regions: Comma separated list of execution region ids to match. Only return files captured in those regions if specified. Note that the regionId can be found in the execution details at regions[i].id. Optional.
  • client: Global user number (starts at 0) to match (globalConcurrentClientIndex in the metadata). Only return files captured by the specified that user if specified. Optional.
  • iteration: Iteration number (starts at 0) to match. Only return files captured during the specified iteration if specified. Optional.
  • failuresOnly: If set to true only return files captured during test iterations that encountered assertion errors. Optional.

Response

[ {
   "id":10566,
   "createdAt":"2020-06-02T05:26:02.695Z",
   "updatedAt":"2020-06-02T05:26:04.149Z",
   "capturedAt":"2020-06-02T05:26:01.941Z",
   "name":"test-0-2.png",
   "type":"image/png",
   "charset":"",
   "size":30373,
   "url":"https://api.testable.io/api/output-files/10566/download",
   "viewUrl":"https://api.testable.io/api/output-files/10566/view",
   "thumbnailUrl":"https://api.testable.io/api/output-files/10566/thumbnail",
   "executionId":3383,
   "executionRegionId":2599,
   "regionId":1,
   "locationId":1,
   "chunkId":2695,
   "chunkIndex":0,
   "concurrentClientIndex":0,
   "globalConcurrentClientIndex":0,
   "regionalConcurrentClientIndex":0,
   "iteration":0,
   "diffPercent":0.38,
   "diffDataStore":{
      "id":8307,
      "createdAt":"2020-06-02T05:23:26.990Z",
      "updatedAt":"2020-06-02T05:23:26.990Z",
      "name":"HomePage.png",
      "type":"image/png",
      "charset":"UTF-8",
      "size":36906,
      "url":"https://api.testable.io/api/data-stores/8307/download",
      "viewUrl":"https://api.testable.io/api/data-stores/8307/view",
      "thumbnailUrl":"https://api.testable.io/api/data-stores/8307/thumbnail",
      "deleteUrl":"https://api.testable.io/api/data-stores/8307",
      "deleteType":"DELETE",
      "rows":[

      ],
      "firstVersion":true
   },
   "expectDiff":true,
   "suiteTestResultId":30122,
   "threadId":"3383:aws-us-east-1:0",
   "iterationId":"c4d880c4-ce73-4f7c-b773-e96361e0c954",
   "isImage":true,
   "isRelatedToFailure":true,
   "agentId":3133,
   "firstVersion":false
} ]

Download zip file of all output files

Download a zip file with all output files from this test run. Structured as follows unless you pass flat=true or the test run is not a load test:

user-0
  iteration-0
    filexxx.txt
    fileyyy.png
    filezzz.jpg
  iteration-1
    ...
user-1
  ...

GET /executions/:execution/output-files.zip

Request Parameters

  • flat: Put all the files at the root of the zip file instead of grouping by user/iteration into folders. Defaults to false for load tests and true for monitors and functional tests.
  • limit: Number of output files to return. Returns all by default or if set to -1.
  • offset: Offset within the result set to start the response. Defaults to 0. When combined with limit it is useful for paging through the results.
  • since: Only return files where capturedAt >= since. Time is specified as a unix epoch in milliseconds (e.g. 1457636340000). Optional.
  • until: Only return files where capturedAt < until. Time is specified as a unix epoch in milliseconds (e.g. 1457636340000). Optional.
  • contentType: Only return files where the content type matches the specified value. Optional.
  • agent: Comma separated list of agent (i.e. test runner) ids to match. Only return files captured from the listed agents if specified. Optional.
  • regions: Comma separated list of execution region ids to match. Only return files captured in those regions if specified. Optional.
  • client: Global user number (starts at 0) to match (globalConcurrentClientIndex in the metadata). Only return files captured by the specified that user if specified. Optional.
  • iteration: Iteration number (starts at 0) to match. Only return files captured during the specified iteration if specified. Optional.
  • failuresOnly: If set to true only return files captured during test iterations that encountered assertion errors. Optional.

Download file by name

Download an output file from your test results by file name. Note that if your test results include multiple files with the same name, the first match will be returned. You can pass the file ID as the fileName parameter to choose a specific match if there are multiple with the same name. File IDs can be found using the GET /executions/:id/output-files endpoint.

GET /executions/:id/output-files/:fileName/download

Export to PDF

Export the aggregated results to a PDF in the same format as you see via the website.

Request Parameters

  • dashboardId: The specific view to use when exporting the results. If you are viewing the results via the website use the “View” dropdown at the top to switch between views. The URL in your browser will update with the dashboardId that corresponds to the selected view. If not specified the default view for the test results is used.
  • tz: The timezone to use when displaying times in the PDF. Defaults to UTC. Use any valid Chrome value (e.g. America/Chicago).
  • enc: By default this endpoint will initiate a file download. If you pass enc=base64 it will instead return the file base64 encoded.
GET /executions/:execution/results.pdf

Download assertions

Download the raw assertion results, unaggregated for this execution. Can also be downloaded for one region where the test executed. Note that the regionId can be found in the execution details at regions[i].id. Omit the .csv at the end to get the results in JSON format.

GET /executions/:execution/suite-results[.csv]
GET /executions/:execution/regions/:regionId/suite-results[.csv]

Get execution intervals

Results are aggregated in 10 second buckets during execution. Can also be retrieved for one region where the test executed. Note that the regionId can be found in the execution details at regions[i].id.

GET /executions/:execution/intervals
GET /executions/:execution/regions/:regionId/intervals

Request Parameters

  • numIntervals: Only return the latest numIntervals intervals based on timestamp. Optional.
  • since: Only return intervals where intervalStart >= since. Time is specified as a unix epoch in milliseconds (e.g. 1457636340000). Optional.

Response

[
   {
      "id":257947,
      "createdAt":"2016-03-10T19:00:56.050Z",
      "updatedAt":"2016-03-10T19:00:56.050Z",
      "executionId":725,
      "intervalStart":1457636450000,
      "iterationsExecuted":3,
      "count":6,
      "concurrentClients":0,
      "actualConcurrentClients":0,
      "maxConcurrentClients":2,
      "metrics":[
         {
            "metricDef":"httpResponseCode",
            "metricType":"Histogram",
            "metricValueMap":{
               "200":3
            }
         },
         {
            "metricDef":"httpMethod",
            "metricType":"Histogram",
            "metricValueMap":{
               "GET":3
            }
         },
         {
            "metricDef":"connectionOpenMs",
            "metricType":"Timing",
            "metricValueMap":{
               "count":3,
               "mean":1,
               "min":1,
               "p95":2,
               "max":2,
               "p99":2,
               "sd":0,
               "p50":2,
               "sum":5,
               "var":0
            },
            "metricUnits":"ms"
         },
         {
            "metricDef":"bandwidth",
            "metricType":"Counter",
            "metricValue":521,
            "metricUnits":"bytes"
         },
         {
            "metricDef":"success",
            "metricType":"Counter",
            "metricValue":6,
            "metricUnits":"requests"
         },
         {
            "metricDef":"dataPacketsSent",
            "metricType":"Counter",
            "metricValue":3,
            "metricUnits":"packets"
         },
         {
            "metricDef":"connectionCloseMs",
            "metricType":"Timing",
            "metricValueMap":{
               "count":3,
               "mean":5,
               "min":4,
               "p95":7,
               "max":7,
               "p99":7,
               "sd":1,
               "p50":6,
               "sum":17,
               "var":2
            },
            "metricUnits":"ms"
         },
         {
            "metricDef":"tickLatencyMs",
            "metricType":"Timing",
            "metricValueMap":{
               "count":3,
               "mean":123,
               "min":123,
               "p95":123,
               "max":123,
               "p99":123,
               "sd":0,
               "p50":123,
               "sum":369,
               "var":0
            },
            "metricUnits":"ms"
         },
         {
            "metricDef":"bytesReceived",
            "metricType":"Counter",
            "metricValue":302,
            "metricUnits":"bytes"
         },
         {
            "metricDef":"dataPacketsReceived",
            "metricType":"Counter",
            "metricValue":6,
            "metricUnits":"packets"
         },
         {
            "metricDef":"firstReceivedMs",
            "metricType":"Timing",
            "metricValueMap":{
               "count":3,
               "mean":5,
               "min":4,
               "p95":7,
               "max":7,
               "p99":7,
               "sd":1,
               "p50":6,
               "sum":17,
               "var":2
            },
            "metricUnits":"ms"
         },
         {
            "metricDef":"bytesSent",
            "metricType":"Counter",
            "metricValue":219,
            "metricUnits":"bytes"
         },
         {
            "metricDef":"dnsLookupMs",
            "metricType":"Timing",
            "metricValueMap":{
               "count":3,
               "mean":0,
               "min":0,
               "p95":1,
               "max":1,
               "p99":1,
               "sd":0,
               "p50":0,
               "sum":1,
               "var":0
            },
            "metricUnits":"ms"
         }
      ]
   }
]

Get execution results by resource

Results are aggregated by resource (e.g. GET http://myserver.com/somethingTested). The resource name is specified during test execution either automatically by Testable or by you when capturing a custom metric.

Can also be retrieved for one region where the test executed. Note that the regionId can be found in the execution details at regions[i].id.

GET /executions/:execution/resources
GET /executions/:execution/regions/:regionId/resources

Request Parameters

  • intervals: Whether or not to include the by interval results for each resource. Defaults to false. Note that this will return a huge amount of data and it is recommended you retrieve interval data for either the entire execution or for one specific resource.

Response

[
   {
      "resource":{
         "id":239,
         "createdAt":"2016-01-31T23:57:52.628Z",
         "updatedAt":"2016-01-31T23:57:52.629Z",
         "resource":"GET http://sample.testable.io/stocks/IBM/stocks/IBM"
      },
      "summary":{
         "id":257943,
         "createdAt":"2016-03-10T19:00:56.030Z",
         "updatedAt":"2016-03-10T19:00:56.030Z",
         "executionId":725,
         "resourceId":239,
         "intervalStart":0,
         "iterationsExecuted":0,
         "count":100,
         "concurrentClients":0,
         "actualConcurrentClients":0,
         "maxConcurrentClients":0,
         "metrics":[
            {
               "metricDef":"httpResponseCode",
               "metricType":"Histogram",
               "metricValueMap":{
                  "200":100
               }
            },
            {
               "metricDef":"httpMethod",
               "metricType":"Histogram",
               "metricValueMap":{
                  "GET":100
               }
            },
            {
               "metricDef":"connectionOpenMs",
               "metricType":"Timing",
               "metricValueMap":{
                  "count":100,
                  "mean":3,
                  "min":1,
                  "p95":17,
                  "max":33,
                  "p99":28,
                  "sd":5,
                  "p50":2,
                  "sum":311,
                  "var":27
               },
               "metricUnits":"ms"
            },
            {
               "metricDef":"bandwidth",
               "metricType":"Counter",
               "metricValue":17264,
               "metricUnits":"bytes"
            },
            {
               "metricDef":"success",
               "metricType":"Counter",
               "metricValue":100,
               "metricUnits":"requests"
            },
            {
               "metricDef":"dataPacketsSent",
               "metricType":"Counter",
               "metricValue":100,
               "metricUnits":"packets"
            },
            {
               "metricDef":"connectionCloseMs",
               "metricType":"Timing",
               "metricValueMap":{
                  "count":100,
                  "mean":6,
                  "min":3,
                  "p95":19,
                  "max":41,
                  "p99":32,
                  "sd":5,
                  "p50":6,
                  "sum":689,
                  "var":31
               },
               "metricUnits":"ms"
            },
            {
               "metricDef":"bytesReceived",
               "metricType":"Counter",
               "metricValue":9964,
               "metricUnits":"bytes"
            },
            {
               "metricDef":"dataPacketsReceived",
               "metricType":"Counter",
               "metricValue":200,
               "metricUnits":"packets"
            },
            {
               "metricDef":"firstReceivedMs",
               "metricType":"Timing",
               "metricValueMap":{
                  "count":100,
                  "mean":6,
                  "min":3,
                  "p95":19,
                  "max":38,
                  "p99":31,
                  "sd":5,
                  "p50":5,
                  "sum":646,
                  "var":29
               },
               "metricUnits":"ms"
            },
            {
               "metricDef":"bytesSent",
               "metricType":"Counter",
               "metricValue":7300,
               "metricUnits":"bytes"
            },
            {
               "metricDef":"dnsLookupMs",
               "metricType":"Timing",
               "metricValueMap":{
                  "count":100,
                  "mean":1,
                  "min":0,
                  "p95":16,
                  "max":31,
                  "p99":27,
                  "sd":5,
                  "p50":1,
                  "sum":193,
                  "var":27
               },
               "metricUnits":"ms"
            }
         ]
      }
   }
]

Get execution results for a resource

Get results for one specific resource. See the get execution results by resource for more explanation on resources. Use that route to get the resource id.

Can also be retrieved for one region where the test executed. Note that the regionId can be found in the execution details at regions[i].id.

GET /executions/:execution/resources/:resourceId
GET /executions/:execution/regions/:regionId/resources/:resourceId

Request Parameters

  • intervals: Whether or not to include the by interval results. Defaults to true.

Response

{
   "resource":{
      "id":239,
      "createdAt":"2016-01-31T23:57:52.628Z",
      "updatedAt":"2016-01-31T23:57:52.629Z",
      "resource":"GET http://sample.testable.io/stocks/IBM/stocks/IBM"
   },
   "intervals": [],
   "summary":{
      "id":257943,
      "createdAt":"2016-03-10T19:00:56.030Z",
      "updatedAt":"2016-03-10T19:00:56.030Z",
      "executionId":725,
      "resourceId":239,
      "intervalStart":0,
      "iterationsExecuted":0,
      "count":100,
      "concurrentClients":0,
      "actualConcurrentClients":0,
      "maxConcurrentClients":0,
      "metrics":[
         {
            "metricDef":"httpResponseCode",
            "metricType":"Histogram",
            "metricValueMap":{
               "200":100
            }
         },
         {
            "metricDef":"httpMethod",
            "metricType":"Histogram",
            "metricValueMap":{
               "GET":100
            }
         },
         {
            "metricDef":"connectionOpenMs",
            "metricType":"Timing",
            "metricValueMap":{
               "count":100,
               "mean":3,
               "min":1,
               "p95":17,
               "max":33,
               "p99":28,
               "sd":5,
               "p50":2,
               "sum":311,
               "var":27
            },
            "metricUnits":"ms"
         },
         {
            "metricDef":"bandwidth",
            "metricType":"Counter",
            "metricValue":17264,
            "metricUnits":"bytes"
         },
         {
            "metricDef":"success",
            "metricType":"Counter",
            "metricValue":100,
            "metricUnits":"requests"
         },
         {
            "metricDef":"dataPacketsSent",
            "metricType":"Counter",
            "metricValue":100,
            "metricUnits":"packets"
         },
         {
            "metricDef":"connectionCloseMs",
            "metricType":"Timing",
            "metricValueMap":{
               "count":100,
               "mean":6,
               "min":3,
               "p95":19,
               "max":41,
               "p99":32,
               "sd":5,
               "p50":6,
               "sum":689,
               "var":31
            },
            "metricUnits":"ms"
         },
         {
            "metricDef":"bytesReceived",
            "metricType":"Counter",
            "metricValue":9964,
            "metricUnits":"bytes"
         },
         {
            "metricDef":"dataPacketsReceived",
            "metricType":"Counter",
            "metricValue":200,
            "metricUnits":"packets"
         },
         {
            "metricDef":"firstReceivedMs",
            "metricType":"Timing",
            "metricValueMap":{
               "count":100,
               "mean":6,
               "min":3,
               "p95":19,
               "max":38,
               "p99":31,
               "sd":5,
               "p50":5,
               "sum":646,
               "var":29
            },
            "metricUnits":"ms"
         },
         {
            "metricDef":"bytesSent",
            "metricType":"Counter",
            "metricValue":7300,
            "metricUnits":"bytes"
         },
         {
            "metricDef":"dnsLookupMs",
            "metricType":"Timing",
            "metricValueMap":{
               "count":100,
               "mean":1,
               "min":0,
               "p95":16,
               "max":31,
               "p99":27,
               "sd":5,
               "p50":1,
               "sum":193,
               "var":27
            },
            "metricUnits":"ms"
         }
      ]
   }
}

Get execution logs

Get all logging captured during execution either in JSON or text format.

GET /executions/:execution/logs[.txt]

Request Parameters

  • limit: Number of log entries to return. Returns all by default or if set to -1.
  • asc: Whether to return the logs in ascending or descending order. true (ascending) by default.
  • systemOnly: Whether to return only system log entries or all log entries including user generating ones. false by default.
  • level: Minimum log level to return: Error, Info, Debug. Returns all logging by default.
  • since: Start timestamp (inclusive) from which to return logs. Specified as unix epoch time in milliseconds. Defaults to the start of the test.
  • until: End timestamp (inclusive) from which to return logs. Specified as unix epoch time in milliseconds. Defaults to the end of the test.
  • client: Each user or engine has a unique zero-based number. If you specify it, only logging from that user/engine will be returned.
  • iteration: Which iteration (zero-based) to return logging from. Only applicable for tools other than Gatling, JMeter, and Locust which are only run one time on each engine.
  • groups: Which logging group(s) (e.g. jmeter.log) to return. For example: jmeter.log, JMeter Console will return the jmeter.log contents and also any stdout/stderr.
  • regions: Which execution regions (by id) to return logging for. You can use GET /executions/:execution to find the execution region ids at execution.regions[*].id.
  • agents: Which test runners (by id) to return logging for. You can use GET /executions/:execution to find the agent ids at execution.agents[*].id.

Response

JSON:

{
   "size":8,
   "logs":[
      {
         "id":38666,
         "level":"Info",
         "msg":"Execution started",
         "length":17,
         "isSystem":true,
         "createdAt":"2016-03-10T18:57:37.069Z",
         "updatedAt":"2016-03-10T18:57:37.091Z",
         "loggedAt":"2016-03-10T18:57:37.069Z"
      },
      {
         "id":38667,
         "level":"Info",
         "msg":"Allocating to agents in AWS N. Virginia",
         "length":39,
         "isSystem":true,
         "createdAt":"2016-03-10T18:57:37.100Z",
         "updatedAt":"2016-03-10T18:57:37.102Z",
         "loggedAt":"2016-03-10T18:57:37.100Z"
      },
      {
         "id":38668,
         "level":"Info",
         "msg":"Allocated to agents in AWS N. Virginia. Testing is underway",
         "length":59,
         "isSystem":true,
         "createdAt":"2016-03-10T18:57:37.160Z",
         "updatedAt":"2016-03-10T18:57:37.162Z",
         "loggedAt":"2016-03-10T18:57:37.160Z"
      },
      {
         "id":38669,
         "level":"Info",
         "msg":"Finished execution in AWS N. Virginia",
         "length":37,
         "isSystem":true,
         "createdAt":"2016-03-10T19:00:55.821Z",
         "updatedAt":"2016-03-10T19:00:55.823Z",
         "loggedAt":"2016-03-10T19:00:55.821Z"
      },
      {
         "id":38670,
         "level":"Info",
         "msg":"Finished execution globally, processing 200 results",
         "length":51,
         "isSystem":true,
         "createdAt":"2016-03-10T19:00:55.846Z",
         "updatedAt":"2016-03-10T19:00:55.848Z",
         "loggedAt":"2016-03-10T19:00:55.846Z"
      },
      {
         "id":38671,
         "level":"Info",
         "msg":"Analyzing and aggregating results",
         "length":33,
         "isSystem":true,
         "createdAt":"2016-03-10T19:00:55.980Z",
         "updatedAt":"2016-03-10T19:00:55.984Z",
         "loggedAt":"2016-03-10T19:00:55.980Z"
      },
      {
         "id":38672,
         "level":"Info",
         "msg":"Finished processing results, cleaning up resources",
         "length":50,
         "isSystem":true,
         "createdAt":"2016-03-10T19:00:56.805Z",
         "updatedAt":"2016-03-10T19:00:56.807Z",
         "loggedAt":"2016-03-10T19:00:56.805Z"
      },
      {
         "id":38673,
         "level":"Info",
         "msg":"Test completed",
         "length":14,
         "isSystem":true,
         "createdAt":"2016-03-10T19:00:56.809Z",
         "updatedAt":"2016-03-10T19:00:56.813Z",
         "loggedAt":"2016-03-10T19:00:56.809Z"
      }
   ]
}

Text:

2016-03-10T18:57:37.069Z INFO Execution started
2016-03-10T18:57:37.100Z INFO Allocating to agents in AWS N. Virginia
2016-03-10T18:57:37.160Z INFO Allocated to agents in AWS N. Virginia. Testing is underway
2016-03-10T19:00:55.821Z INFO Finished execution in AWS N. Virginia
2016-03-10T19:00:55.846Z INFO Finished execution globally, processing 200 results
2016-03-10T19:00:55.980Z INFO Analyzing and aggregating results
2016-03-10T19:00:56.805Z INFO Finished processing results, cleaning up resources
2016-03-10T19:00:56.809Z INFO Test completed

Get execution traces

Get all connection traces captured as part of this execution. Traces allow you to see exactly what happened during a connection including metrics, data sent, and data received.

Can also be retrieved for one region where the test executed. Note that the regionId can be found in the execution details at regions[i].id.

GET /executions/:execution/traces
GET /executions/:execution/regions/:regionId/traces

Request Parameters

  • limit: Number of traces to return. Returns all by default or if set to -1.
  • offset: Offset within the results to start the response. Defaults to 0. When combined with limit it is useful for paging through the results.

Response

[
   {
      "id":430,
      "length":219,
      "createdAt":"2016-03-19T19:42:36.889Z",
      "updatedAt":"2016-03-19T19:42:36.889Z",
      "traceAt":"2016-03-19T19:42:32.466Z",
      "status":"301",
      "regionId":857,
      "resource":"GET http://google.com/"
   },
   {
      "id":431,
      "length":0,
      "createdAt":"2016-03-19T19:42:37.471Z",
      "updatedAt":"2016-03-19T19:42:37.471Z",
      "traceAt":"2016-03-19T19:42:32.477Z",
      "status":"204",
      "regionId":857,
      "resource":"GET http://www.google.com/client_204?&atyp=i&biw=400&bih=300&ei=92LgVrP-Bszb-QHPlZXoCA"
   }
]

Get execution trace

Get all details of a trace captured as part of this execution. Traces allow you to see exactly what happened during a connection including metrics, data sent, and data received.

Packet data is base64 encoded. In Javascript it can be decoded using atob(data).

GET /execution-traces/:id

Request Parameters

  • up: Whether or not to return the parent entity, the execution region in this case. Defaults to true.
  • down: Whether or not to return the child entities, the trace packets in this case. Defaults to true.
  • includeResult: Whether or not to return the result metrics captured during this connection. Defaults to true.

Response

{
   "id":430,
   "length":219,
   "createdAt":"2016-03-19T19:42:36.889Z",
   "updatedAt":"2016-03-19T19:42:36.889Z",
   "traceAt":"2016-03-19T19:42:32.466Z",
   "status":"301",
   "regionId":857,
   "resource":"GET http://google.com/",
   "result":{
      "id":1455678,
      "createdAt":"2016-03-19T19:42:36.763Z",
      "updatedAt":"2016-03-19T19:42:36.767Z",
      "executionId":809,
      "executionRegionId":857,
      "chunkId":813,
      "connectionId":"5ab973d0-e828-4049-a03b-56017710c5b8",
      "agentId":"3cd00a3a-0387-42a0-be1d-85d6899008d1",
      "threadId":"86ebef43-11a2-400e-8f60-48729000e39f",
      "timestamp":"2016-03-19T19:42:32.466Z",
      "resource":{
         "id":1,
         "createdAt":"2015-08-11T17:06:34.780Z",
         "updatedAt":"2015-08-11T17:06:34.781Z",
         "resource":"GET http://google.com/"
      },
      "metrics":[
         {
            "metricDef":"dnsLookupMs",
            "metricType":"Timing",
            "metricValue":19,
            "metricUnits":"ms"
         },
         {
            "metricDef":"connectionOpenMs",
            "metricType":"Timing",
            "metricValue":22,
            "metricUnits":"ms"
         },
         {
            "metricDef":"firstReceivedMs",
            "metricType":"Timing",
            "metricValue":47,
            "metricUnits":"ms"
         },
         {
            "metricDef":"httpMethod",
            "metricType":"Histogram",
            "metricKey":"GET",
            "metricValue":1
         },
         {
            "metricDef":"dataPacketsReceived",
            "metricType":"Counter",
            "metricValue":2,
            "metricUnits":"packets"
         },
         {
            "metricDef":"httpResponseCode",
            "metricType":"Histogram",
            "metricKey":"301",
            "metricValue":1
         },
         {
            "metricDef":"success",
            "metricType":"Counter",
            "metricValue":1,
            "metricUnits":"requests"
         },
         {
            "metricDef":"connectionCloseMs",
            "metricType":"Timing",
            "metricValue":50,
            "metricUnits":"ms"
         },
         {
            "metricDef":"bytesSent",
            "metricType":"Counter",
            "metricValue":300,
            "metricUnits":"bytes"
         },
         {
            "metricDef":"bytesReceived",
            "metricType":"Counter",
            "metricValue":540,
            "metricUnits":"bytes"
         },
         {
            "metricDef":"bandwidth",
            "metricType":"Counter",
            "metricValue":840,
            "metricUnits":"bytes"
         }
      ]
   },
   "packets":[
      {
         "id":2370,
         "length":0,
         "packetType":"DnsLookup",
         "dataLength":0,
         "createdAt":"2016-03-19T19:42:36.896Z",
         "updatedAt":"2016-03-19T19:42:36.896Z",
         "packetAt":"2016-03-19T19:42:32.484Z",
         "lastPacketAt":"2016-03-19T19:42:32.484Z"
      },
      {
         "id":2371,
         "length":0,
         "packetType":"ConnectionOpened",
         "dataLength":0,
         "createdAt":"2016-03-19T19:42:36.896Z",
         "updatedAt":"2016-03-19T19:42:36.896Z",
         "packetAt":"2016-03-19T19:42:32.488Z",
         "lastPacketAt":"2016-03-19T19:42:32.488Z"
      },
      {
         "id":2372,
         "length":0,
         "packetType":"DataSent",
         "headers":"{\"User-Agent\":\"Mozilla/5.0 (Unknown; Linux x86_64) AppleWebKit/534.34 (KHTML, like Gecko) PhantomJS/1.9.7 Safari/534.34\",\"Accept\":\"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\",\"Connection\":\"Keep-Alive\",\"Accept-Encoding\":\"gzip\",\"Accept-Language\":\"en-US,*\",\"Host\":\"google.com\"}",
         "dataLength":0,
         "createdAt":"2016-03-19T19:42:36.896Z",
         "updatedAt":"2016-03-19T19:42:36.896Z",
         "packetAt":"2016-03-19T19:42:32.488Z",
         "lastPacketAt":"2016-03-19T19:42:32.488Z"
      },
      {
         "id":2373,
         "length":219,
         "packetType":"DataReceived",
         "headers":"{\"location\":\"http://www.google.com/\",\"content-type\":\"text/html; charset=UTF-8\",\"date\":\"Sat, 19 Mar 2016 19:42:32 GMT\",\"expires\":\"Mon, 18 Apr 2016 19:42:32 GMT\",\"cache-control\":\"public, max-age=2592000\",\"server\":\"gws\",\"content-length\":\"219\",\"x-xss-protection\":\"1; mode=block\",\"x-frame-options\":\"SAMEORIGIN\"}",
         "data":"PEhUTUw+PEhFQUQ+PG1ldGEgaHR0cC1lcXVpdj0iY29udGVudC10eXBlIiBjb250ZW50PSJ0ZXh0L2h0bWw7Y2hhcnNldD11dGYtOCI+CjxUSVRMRT4zMDEgTW92ZWQ8L1RJVExFPjwvSEVBRD48Qk9EWT4KPEgxPjMwMSBNb3ZlZDwvSDE+ClRoZSBkb2N1bWVudCBoYXMgbW92ZWQKPEEgSFJFRj0iaHR0cDovL3d3dy5nb29nbGUuY29tLyI+aGVyZTwvQT4uDQo8L0JPRFk+PC9IVE1MPg0K",
         "dataLength":219,
         "createdAt":"2016-03-19T19:42:36.896Z",
         "updatedAt":"2016-03-19T19:42:36.896Z",
         "packetAt":"2016-03-19T19:42:32.513Z",
         "lastPacketAt":"2016-03-19T19:42:32.515Z"
      },
      {
         "id":2374,
         "length":0,
         "packetType":"ConnectionClosed",
         "dataLength":0,
         "createdAt":"2016-03-19T19:42:36.896Z",
         "updatedAt":"2016-03-19T19:42:36.896Z",
         "packetAt":"2016-03-19T19:42:32.517Z",
         "lastPacketAt":"2016-03-19T19:42:32.517Z"
      }
   ]
}

Start execution

Starts a new execution of the test configuration.

POST /test-confs/:confId/executions

Request Parameters

None. All configuration parameters are captured when creating the test configuration.

Response

{
   "id":735,
   "createdAt":"2016-03-12T05:15:38.160Z",
   "updatedAt":"2016-03-12T05:15:38.179Z",
   "running":false,
   "allocated":false,
   "cancelledByUser":false,
   "completed":false,
   "summaryCalculated":false,
   "limitBreached":false,
   "iterations":20,
   "iterationSleepSecs":10,
   "rampupSecs":0,
   "regions":[
      {
         "id":773,
         "createdAt":"2016-03-12T05:15:38.192Z",
         "updatedAt":"2016-03-12T05:15:38.192Z",
         "running":false,
         "completed":false,
         "concurrentClients":5,
         "actualConcurrentClients":5,
         "expectedResultCount":100,
         "region":{
            "id":1,
            "createdAt":"2015-08-11T17:03:34.761Z",
            "updatedAt":"2015-08-11T17:03:34.761Z",
            "name":"aws-us-east-1",
            "public":true,
            "latitude":39.0436,
            "longitude":-77.4878,
            "description":"AWS N. Virginia",
            "active":true
         }
      }
   ],
   "concurrentClients":5,
   "actualConcurrentClients":5,
   "expectedResultCount":100,
   "scriptParams":"{}",
   "script":{
      "id":273,
      "createdAt":"2016-02-18T19:00:46.862Z",
      "updatedAt":"2016-03-09T08:10:44.945Z",
      "name":"Default Scenario",
      "length":97,
      "hasInit":false,
      "hasTeardown":false,
      "scriptType":"Code"
   }
}

Check if execution exists

HEAD /executions/:id

Response

HTTP Status: 200 OK or 404 NOT FOUND

Start execution

Request to start the execution. Note that creating and starting an execution are two steps.

PUT /executions/:id/start

Request Parameters

None

Response

{
   "updatedAt":"2016-03-12T05:15:41.830Z",
   "iterationSleepSecs":10,
   "allocated":false,
   "limitBreached":false,
   "running":true,
   "summaryCalculated":false,
   "rampupSecs":0,
   "actualConcurrentClients":5,
   "regions":[
      {
         "updatedAt":"2016-03-12T05:15:38.192Z",
         "running":false,
         "actualConcurrentClients":5,
         "id":773,
         "completed":false,
         "createdAt":"2016-03-12T05:15:38.192Z",
         "region":{
            "name":"aws-us-east-1",
            "updatedAt":"2015-08-11T17:03:34.761Z",
            "latitude":39.0436,
            "description":"AWS N. Virginia",
            "public":true,
            "longitude":-77.4878,
            "id":1,
            "createdAt":"2015-08-11T17:03:34.761Z",
            "active":true
         },
         "concurrentClients":5,
         "expectedResultCount":100
      }
   ],
   "id":735,
   "completed":false,
   "scriptParams":"{}",
   "cancelledByUser":false,
   "createdAt":"2016-03-12T05:15:38.160Z",
   "startedAt":"2016-03-12T05:15:41.829Z",
   "script":{
      "name":"Default Scenario",
      "updatedAt":"2016-03-09T08:10:44.945Z",
      "hasTeardown":false,
      "scriptType":"Code",
      "id":273,
      "createdAt":"2016-02-18T19:00:46.862Z",
      "length":97,
      "hasInit":false
   },
   "iterations":20,
   "concurrentClients":5,
   "expectedResultCount":100
}

Manual start

Tests that were started with manual_start=true will simply sit and wait after the test runners are allocated and initialized. This route tells Testable to start executing the test across the test runners. This can be useful with on-demand test runners since we don’t know exactly how long AWS EC2 instances take to spin up. Start the test with manual_start=true well in advance of your desired test run time and when you trigger the manual start it will immediately start generating load.

Stop execution

Request to stop the execution before it completes. Note that once the execution is stopped it will take a little while before it has completed: true status due to cleanup, result aggregation, etc.

PUT /executions/:id/stop

Request Parameters

None

Response

{
   "updatedAt":"2016-03-12T05:15:41.830Z",
   "iterationSleepSecs":10,
   "allocated":false,
   "limitBreached":false,
   "running":false,
   "summaryCalculated":false,
   "rampupSecs":0,
   "actualConcurrentClients":5,
   "regions":[
      {
         "updatedAt":"2016-03-12T05:15:38.192Z",
         "running":false,
         "actualConcurrentClients":5,
         "id":773,
         "completed":false,
         "createdAt":"2016-03-12T05:15:38.192Z",
         "region":{
            "name":"aws-us-east-1",
            "updatedAt":"2015-08-11T17:03:34.761Z",
            "latitude":39.0436,
            "description":"AWS N. Virginia",
            "public":true,
            "longitude":-77.4878,
            "id":1,
            "createdAt":"2015-08-11T17:03:34.761Z",
            "active":true
         },
         "concurrentClients":5,
         "expectedResultCount":100
      }
   ],
   "id":735,
   "completed":false,
   "scriptParams":"{}",
   "cancelledByUser":true,
   "createdAt":"2016-03-12T05:15:38.160Z",
   "startedAt":"2016-03-12T05:15:41.829Z",
   "finishedAt":"2016-03-12T05:15:42.736Z",
   "script":{
      "name":"Default Scenario",
      "updatedAt":"2016-03-09T08:10:44.945Z",
      "hasTeardown":false,
      "scriptType":"Code",
      "id":273,
      "createdAt":"2016-02-18T19:00:46.862Z",
      "length":97,
      "hasInit":false
   },
   "iterations":20,
   "concurrentClients":5,
   "expectedResultCount":100
}

Delete execution

Delete this execution. Only works once the execution is completed. If you want to delete before it completes, please stop first and wait until the completed field is true.

DELETE /executions/:id

Response

The deleted execution

{
   "updatedAt":"2016-03-12T05:15:41.830Z",
   "iterationSleepSecs":10,
   "allocated":true,
   "limitBreached":false,
   "running":false,
   "summaryCalculated":true,
   "rampupSecs":0,
   "actualConcurrentClients":5,
   "regions":[
      {
         "updatedAt":"2016-03-12T05:15:38.192Z",
         "running":false,
         "actualConcurrentClients":5,
         "id":773,
         "completed":true,
         "createdAt":"2016-03-12T05:15:38.192Z",
         "region":{
            "name":"aws-us-east-1",
            "updatedAt":"2015-08-11T17:03:34.761Z",
            "latitude":39.0436,
            "description":"AWS N. Virginia",
            "public":true,
            "longitude":-77.4878,
            "id":1,
            "createdAt":"2015-08-11T17:03:34.761Z",
            "active":true
         },
         "concurrentClients":5,
         "expectedResultCount":100
      }
   ],
   "id":735,
   "completed":true,
   "scriptParams":"{}",
   "cancelledByUser":true,
   "createdAt":"2016-03-12T05:15:38.160Z",
   "startedAt":"2016-03-12T05:15:41.829Z",
   "finishedAt":"2016-03-12T05:15:42.736Z",
   "script":{
      "name":"Default Scenario",
      "updatedAt":"2016-03-09T08:10:44.945Z",
      "hasTeardown":false,
      "scriptType":"Code",
      "id":273,
      "createdAt":"2016-02-18T19:00:46.862Z",
      "length":97,
      "hasInit":false
   },
   "iterations":20,
   "concurrentClients":5,
   "expectedResultCount":100
}

Extend execution

Changes the duration or number of iterations this test will execute for. Propogates live to all test runners currently running the test. If the new duration/iterations has already passed the test will complete once the currently executing iteration is completed.

This only works for scenario types where Testable manages the concurrency (i.e. not JMeter or Gatling).

POST /executions/:executionId/live-extension

Request Body

For tests that are configured to run for a duration (in seconds):

{
   "newDuration": 360 
}

For tests that are configured to run for a certain number of iterations:

{
   "newIterations": 5
}

Response

200 OK status

Add users/engines

Adds additional virtual users or engines (for JMeter/Gatling/Locust) on the fly while the test is running. If the test is utilizing per-test runners, additional runners may be provisioned first.

POST /executions/:executionId/add-users

Request Body

{
   "users": 5,
   "executionRegion": 12345
}
  • users: Number of virtual users or engines to add. For JMeter/Gatling/Locust this number corresponds to the number of engines to add.
  • executionRegion: Which region to add the users. The execution region id can be found using the GET /executions/:id API at regions[*].id in the response.

Response

200 OK status

Send event

Send a live event to all virtual users while the test is running

POST /executions/:executionId/send-event

Request Body

The name of the event (required) and any contents to pass along (optional).

{
   "name": "load-url",
   "contents": "https://www.amazon.com" 
}

Response

200 OK status

Share results

Share this test result via a public share link. Anyone who has the link can see the results without logging in.

POST /executions/:executionId/share

Request Body

Optionally specify the view id and/or timezone that you want people to see the results with when accessing the share link

{
   "dashboardId": 0,
   "tz": "America/Chicago" 
}

Response

{
   "id":1773,
   "createdAt":"2022-12-12T22:52:08.400Z",
   "updatedAt":"2022-12-12T22:52:08.400Z",
   "name":"rprimspglm",
   "targetType":"GetLoadTestExecution",
   "targetId":15450,
   "description":"{\"dashboardId\":0,\"tz\":\"America/Chicago\"}",
   "active":true,
   "organization":{
      "id":2
   },
   "firstVersion":true
}

The name property can then be used as the public token via https://a.testable.io/p/:name to access the test results. Use https://a.testable.io/p/:name/results.pdf to download the results as a PDF.

Tag results

Apply one or more tags to a test result. These tags are useful for grouping results together or for looking up a test result later by an external identifier.

POST /executions/:id/tags

Request Body

{
   "names": [ "tag1", "tag2" ]
}

Response

[
    {
        "id": 273,
        "createdAt": "2023-04-18T22:09:56.768Z",
        "updatedAt": "2023-04-18T22:09:56.768Z",
        "name": "tag1",
        "firstVersion": true
    },
    {
        "id": 274,
        "createdAt": "2023-04-18T22:09:56.768Z",
        "updatedAt": "2023-04-18T22:09:56.768Z",
        "name": "tag2",
        "firstVersion": true
    }
]

Search for results

Search for test results using one or more filters. The different types of filters are ANDed together (e.g. tagNames IN (“tag1”,”tag2”) AND testCases IN (1,2,3)).

POST /search
POST /orgs/:org/search

Request Parameters

  • offset: For paging results. The offset within the results to start at. Defaults to 0.
  • limit: For paging results. Limit on how many results to return. Defaults to 20.

Request Body

All fields in the filter are optional. If an empty body is passed, all test results

{
   "tagNames": [ "tag1", "tag2" ], # tag names
   "testCases": [1, 2, 3], # test case id
   "configurations": [1, 2, 3], #configuration ids
   "scenarios": [1, 2, 3], #scenario ids
   "outcome": "success|failure", # success for just successful results, failure for just failed results
   "timeRange": { 
      "from": 1681859412000, # unix epoch in ms
      "to": 1681858412000, # unix epoch in ms
      "relativeTime": "5d" # ms|s|m|h|d|weeks|months are the possible units
   }
   "metrics": [
      {
         "namespace": "Testable",
         "metric": "outcome", # histogram example: % of requests successful > 95%
         "metricKey": "success",
         "threshold": 95,
         "operator": "GreaterThan", # GTE, GreaterThan, LTE, LessThan
         "isPercentage": true # whether to query on count in bucket or percentage of total in histogram
      },
      {
         "namespace": "Testable",
         "metric": "firstReceivedMs", # timing example: median response time >= 1000ms
         "metricKey": "p50",
         "threshold": 1000,
         "operator": "GTE", # GTE, GreaterThan, LTE, LessThan
      },
      {
         "namespace": "Testable",
         "metric": "bandwidth", # counter example: total bandwidth >= 1MB
         "threshold": 1048576,
         "operator": "GTE", # GTE, GreaterThan, LTE, LessThan
      }
   ],
   "testType": [ "Load" ] # Load, Functional, Monitor, Remote, Live
}

Response

[
   {
      "id": 12345,
      ...
   }
]

Evaluate Test Result

Evaluate test results against provided KPIs. The list of KPIs are ANDed together for the evaluation.

POST /executions/:id/evaluate-kpis

Request Body

[
  {
    "expr": "Outcome[success] > 98%",
    "type": "value"
  }
]

For more details on supported values for expr and type fields, refer parameter - KPIs in the Simple Test API doc.
Response

{
  "success": true
}