Route | Description |
---|---|
GET /test-confs/:confId/executions | Get list of executions |
GET /executions/:id | Get execution details |
GET /executions/:id/summary | Get execution summary |
GET /executions/:id/metric | Get execution metric |
GET /executions/:id/results[.csv] | Download execution results |
GET /executions/:id/suite-results[.csv] | Download assertion results |
GET /executions/:id/suite-summaries[.csv] | Download assertion summary |
GET /executions/:id/suite-errors[.csv] | Download assertion error summary |
GET /executions/:id/traces.zip | Download traces |
GET /executions/:id/output-files | Get output file metadata |
GET /executions/:id/output-files.zip | Download zip file of all output files |
GET /executions/:id/output-files/:fileName/download | Download file by name or id |
GET /executions/:id/results.pdf | Export to PDF |
GET /executions/:id/suite-results[.csv] | Download assertion results |
GET /executions/:id/intervals | Get execution intervals |
GET /executions/:id/resources | Get execution results by resource |
GET /executions/:id/resources/:resourceId | Get execution results for a resource |
GET /executions/:id/logs[.txt] | Get execution logs |
GET /executions/:id/traces | Get execution traces |
GET /execution-traces/:id | Get execution trace |
POST /test-confs/:confId/executions | Start execution |
HEAD /executions/:id | Check if execution exists |
PUT /executions/:id/manual-start | Manual start |
PUT /executions/:id/stop | Stop execution |
DELETE /executions/:id | Delete execution |
POST /executions/:id/add-users | Add users/engines |
POST /executions/:id/live-extension | Extend execution |
POST /executions/:id/send-event | Send event |
POST /orgs/:orgId/tokens | Share results |
Overview
API routes related to test executions. A test execution is a load test that runs a scenario under a certain configuration. Each execution contains one or more regions where it executes. Within each execution region, the work is split up into “chunks”. Test execution results are aggregated in 10 second intervals. Result metrics are grouped by resource.
Get list of executions
Retrieve a list of all executions within a given test configuration.
GET /test-confs/:confId/executions
Request Parameters
down
: Whether or not to include execution region/chunk information. Defaults to true.up
: Whether or not to include the parent details. Defaults to false.includeScript
: Whether or not to include the script information used to run the test. Defaults to true.includeSummary
: Whether or not to include the metric summary. Defaults to true.includeResources
: Whether or not to include the by resource results. Defaults to false.offset
: Offset within the executions to start the response. Defaults to 0. When combined withlimit
it is useful for paging through the results.period
: What time period to query past executions in. Possible values arerecent
(last 15 orlimit
parameter value),7d
(last 7 days),1m
(last 1 month),3m
(last 3 months). Defaults torecent
if start/end not specified.start
: The startcreatedAt
timestamp (Unix epoch in milliseconds, UTC). If specified, theperiod
parameter is ignored.end
: The endcreatedAt
timestamp (Unix epoch in milliseconds, UTC). Defaults to the current timestamp ifstart
is specified.limit
: The number of results to return. Defaults to 15 unless a period other thanrecent
is specified in which case it defaults to -1 (i.e. return entire history).
Sample Response
[
{
"id":637,
"createdAt":"2016-02-18T19:11:52.737Z",
"updatedAt":"2016-02-18T19:15:26.544Z",
"running":false,
"allocated":true,
"startedAt":"2016-02-18T19:11:54.095Z",
"finishedAt":"2016-02-18T19:15:25.150Z",
"cancelledByUser":false,
"completed":true,
"summaryCalculated":true,
"limitBreached":false,
"iterations":20,
"iterationSleepSecs":10,
"rampupSecs":0,
"regions":[
{
"id":669,
"createdAt":"2016-02-18T19:11:52.744Z",
"updatedAt":"2016-02-18T19:15:25.130Z",
"running":false,
"startedAt":"2016-02-18T19:12:06.045Z",
"finishedAt":"2016-02-18T19:15:25.130Z",
"completed":true,
"concurrentClients":5,
"actualConcurrentClients":5,
"expectedResultCount":100,
"region":{
"id":1,
"createdAt":"2015-08-11T17:03:34.761Z",
"updatedAt":"2015-08-11T17:03:34.761Z",
"name":"aws-us-east-1",
"public":true,
"latitude":39.0436,
"longitude":-77.4878,
"description":"AWS N. Virginia",
"active":true
},
"summary":{
"id":206865,
"createdAt":"2016-02-18T19:15:25.749Z",
"updatedAt":"2016-02-18T19:15:25.749Z",
"executionId":637,
"executionRegionId":669,
"intervalStart":0,
"iterationsExecuted":100,
"count":100,
"concurrentClients":5,
"actualConcurrentClients":0,
"maxConcurrentClients":0,
"metrics":[
{
"metricDef":"httpResponseCode",
"metricType":"Histogram",
"metricValueMap":{
"200":100
}
},
{
"metricDef":"connectionOpenMs",
"metricType":"Timing",
"metricValueMap":{
"count":100,
"mean":5,
"min":1,
"p95":9,
"max":214,
"p99":41,
"sd":21,
"sum":527,
"var":481
},
"metricUnits":"ms"
},
{
"metricDef":"bandwidth",
"metricType":"Counter",
"metricValue":17261,
"metricUnits":"bytes"
},
{
"metricDef":"success",
"metricType":"Counter",
"metricValue":100,
"metricUnits":"requests"
},
{
"metricDef":"dataPacketsSent",
"metricType":"Counter",
"metricValue":100,
"metricUnits":"packets"
},
{
"metricDef":"connectionCloseMs",
"metricType":"Timing",
"metricValueMap":{
"count":100,
"mean":8,
"min":3,
"p95":12,
"max":221,
"p99":43,
"sd":22,
"sum":886,
"var":497
},
"metricUnits":"ms"
},
{
"metricDef":"bytesReceived",
"metricType":"Counter",
"metricValue":9961,
"metricUnits":"bytes"
},
{
"metricDef":"dataPacketsReceived",
"metricType":"Counter",
"metricValue":200,
"metricUnits":"packets"
},
{
"metricDef":"firstReceivedMs",
"metricType":"Timing",
"metricValueMap":{
"count":100,
"mean":8,
"min":3,
"p95":11,
"max":217,
"p99":43,
"sd":21,
"sum":847,
"var":482
},
"metricUnits":"ms"
},
{
"metricDef":"bytesSent",
"metricType":"Counter",
"metricValue":7300,
"metricUnits":"bytes"
},
{
"metricDef":"dnsLookupMs",
"metricType":"Timing",
"metricValueMap":{
"count":100,
"mean":4,
"min":0,
"p95":8,
"max":212,
"p99":40,
"sd":21,
"sum":406,
"var":477
},
"metricUnits":"ms"
}
]
},
"chunks":[
{
"id":638,
"executionType":"Main",
"agent":"7316fdd5-a620-401f-ae3a-9c9468489ed4",
"createdAt":"2016-02-18T19:12:06.082Z",
"updatedAt":"2016-02-18T19:15:25.111Z",
"running":false,
"startedAt":"2016-02-18T19:12:06.499Z",
"finishedAt":"2016-02-18T19:15:25.111Z",
"completed":true,
"iterations":20,
"iterationSleepSecs":10,
"rampupSecs":0,
"concurrentClients":5,
"actualConcurrentClients":5,
"completedDurationSecs":0,
"completedIterations":0
}
]
}
],
"concurrentClients":5,
"actualConcurrentClients":5,
"expectedResultCount":100,
"scriptParams":"{}",
"summary":{
"id":206866,
"createdAt":"2016-02-18T19:15:25.749Z",
"updatedAt":"2016-02-18T19:15:25.749Z",
"executionId":637,
"intervalStart":0,
"iterationsExecuted":100,
"count":100,
"concurrentClients":5,
"actualConcurrentClients":0,
"maxConcurrentClients":0,
"metrics":[
{
"metricDef":"httpResponseCode",
"metricType":"Histogram",
"metricValueMap":{
"200":100
}
},
{
"metricDef":"connectionOpenMs",
"metricType":"Timing",
"metricValueMap":{
"count":100,
"mean":5,
"min":1,
"p95":9,
"max":214,
"p99":41,
"sd":21,
"sum":527,
"var":481
},
"metricUnits":"ms"
},
{
"metricDef":"bandwidth",
"metricType":"Counter",
"metricValue":17261,
"metricUnits":"bytes"
},
{
"metricDef":"success",
"metricType":"Counter",
"metricValue":100,
"metricUnits":"requests"
},
{
"metricDef":"dataPacketsSent",
"metricType":"Counter",
"metricValue":100,
"metricUnits":"packets"
},
{
"metricDef":"connectionCloseMs",
"metricType":"Timing",
"metricValueMap":{
"count":100,
"mean":8,
"min":3,
"p95":12,
"max":221,
"p99":43,
"sd":22,
"sum":886,
"var":497
},
"metricUnits":"ms"
},
{
"metricDef":"bytesReceived",
"metricType":"Counter",
"metricValue":9961,
"metricUnits":"bytes"
},
{
"metricDef":"dataPacketsReceived",
"metricType":"Counter",
"metricValue":200,
"metricUnits":"packets"
},
{
"metricDef":"firstReceivedMs",
"metricType":"Timing",
"metricValueMap":{
"count":100,
"mean":8,
"min":3,
"p95":11,
"max":217,
"p99":43,
"sd":21,
"sum":847,
"var":482
},
"metricUnits":"ms"
},
{
"metricDef":"bytesSent",
"metricType":"Counter",
"metricValue":7300,
"metricUnits":"bytes"
},
{
"metricDef":"dnsLookupMs",
"metricType":"Timing",
"metricValueMap":{
"count":100,
"mean":4,
"min":0,
"p95":8,
"max":212,
"p99":40,
"sd":21,
"sum":406,
"var":477
},
"metricUnits":"ms"
}
]
},
"script":{
"id":273,
"createdAt":"2016-02-18T19:00:46.862Z",
"updatedAt":"2016-03-09T08:10:44.945Z",
"name":"Default Scenario",
"length":97,
"hasInit":false,
"hasTeardown":false,
"scriptType":"Code"
}
},
...
]
Get execution details
Retrieve details of a single execution. If the completed
field is not true
, then the test is still running.
GET /executions/:id
Request Parameters
down
: Whether or not to include execution region/chunk information. Defaults to true.up
: Whether or not to include the parent details. Defaults to true.includeScript
: Whether or not to include the script information used to run the test. Defaults to true.includeSummary
: Whether or not to include the metric summary. Defaults to true.includeIntervals
: Whether or not to include the by interval results. Defaults to false.includeResources
: Whether or not to include the by resource results. Defaults to false.
Sample Response
{
"id":637,
"createdAt":"2016-02-18T19:11:52.737Z",
"updatedAt":"2016-02-18T19:15:26.544Z",
"running":false,
"allocated":true,
"startedAt":"2016-02-18T19:11:54.095Z",
"finishedAt":"2016-02-18T19:15:25.150Z",
"cancelledByUser":false,
"completed":true,
"summaryCalculated":true,
"limitBreached":false,
"iterations":20,
"iterationSleepSecs":10,
"rampupSecs":0,
"regions":[
{
"id":669,
"createdAt":"2016-02-18T19:11:52.744Z",
"updatedAt":"2016-02-18T19:15:25.130Z",
"running":false,
"startedAt":"2016-02-18T19:12:06.045Z",
"finishedAt":"2016-02-18T19:15:25.130Z",
"completed":true,
"concurrentClients":5,
"actualConcurrentClients":5,
"expectedResultCount":100,
"region":{
"id":1,
"createdAt":"2015-08-11T17:03:34.761Z",
"updatedAt":"2015-08-11T17:03:34.761Z",
"name":"aws-us-east-1",
"public":true,
"latitude":39.0436,
"longitude":-77.4878,
"description":"AWS N. Virginia",
"active":true
},
"summary":{
"id":206865,
"createdAt":"2016-02-18T19:15:25.749Z",
"updatedAt":"2016-02-18T19:15:25.749Z",
"executionId":637,
"executionRegionId":669,
"intervalStart":0,
"iterationsExecuted":100,
"count":100,
"concurrentClients":5,
"actualConcurrentClients":0,
"maxConcurrentClients":0,
"metrics":[
{
"metricDef":"httpResponseCode",
"metricType":"Histogram",
"metricValueMap":{
"200":100
}
},
{
"metricDef":"connectionOpenMs",
"metricType":"Timing",
"metricValueMap":{
"count":100,
"mean":5,
"min":1,
"p95":9,
"max":214,
"p99":41,
"sd":21,
"sum":527,
"var":481
},
"metricUnits":"ms"
},
{
"metricDef":"bandwidth",
"metricType":"Counter",
"metricValue":17261,
"metricUnits":"bytes"
},
{
"metricDef":"success",
"metricType":"Counter",
"metricValue":100,
"metricUnits":"requests"
},
{
"metricDef":"dataPacketsSent",
"metricType":"Counter",
"metricValue":100,
"metricUnits":"packets"
},
{
"metricDef":"connectionCloseMs",
"metricType":"Timing",
"metricValueMap":{
"count":100,
"mean":8,
"min":3,
"p95":12,
"max":221,
"p99":43,
"sd":22,
"sum":886,
"var":497
},
"metricUnits":"ms"
},
{
"metricDef":"bytesReceived",
"metricType":"Counter",
"metricValue":9961,
"metricUnits":"bytes"
},
{
"metricDef":"dataPacketsReceived",
"metricType":"Counter",
"metricValue":200,
"metricUnits":"packets"
},
{
"metricDef":"firstReceivedMs",
"metricType":"Timing",
"metricValueMap":{
"count":100,
"mean":8,
"min":3,
"p95":11,
"max":217,
"p99":43,
"sd":21,
"sum":847,
"var":482
},
"metricUnits":"ms"
},
{
"metricDef":"bytesSent",
"metricType":"Counter",
"metricValue":7300,
"metricUnits":"bytes"
},
{
"metricDef":"dnsLookupMs",
"metricType":"Timing",
"metricValueMap":{
"count":100,
"mean":4,
"min":0,
"p95":8,
"max":212,
"p99":40,
"sd":21,
"sum":406,
"var":477
},
"metricUnits":"ms"
}
]
},
"chunks":[
{
"id":638,
"executionType":"Main",
"agent":"7316fdd5-a620-401f-ae3a-9c9468489ed4",
"createdAt":"2016-02-18T19:12:06.082Z",
"updatedAt":"2016-02-18T19:15:25.111Z",
"running":false,
"startedAt":"2016-02-18T19:12:06.499Z",
"finishedAt":"2016-02-18T19:15:25.111Z",
"completed":true,
"iterations":20,
"iterationSleepSecs":10,
"rampupSecs":0,
"concurrentClients":5,
"actualConcurrentClients":5,
"completedDurationSecs":0,
"completedIterations":0
}
]
}
],
"concurrentClients":5,
"actualConcurrentClients":5,
"expectedResultCount":100,
"scriptParams":"{}",
"summary":{
"id":206866,
"createdAt":"2016-02-18T19:15:25.749Z",
"updatedAt":"2016-02-18T19:15:25.749Z",
"executionId":637,
"intervalStart":0,
"iterationsExecuted":100,
"count":100,
"concurrentClients":5,
"actualConcurrentClients":0,
"maxConcurrentClients":0,
"metrics":[
{
"metricDef":"httpResponseCode",
"metricType":"Histogram",
"metricValueMap":{
"200":100
}
},
{
"metricDef":"connectionOpenMs",
"metricType":"Timing",
"metricValueMap":{
"count":100,
"mean":5,
"min":1,
"p95":9,
"max":214,
"p99":41,
"sd":21,
"sum":527,
"var":481
},
"metricUnits":"ms"
},
{
"metricDef":"bandwidth",
"metricType":"Counter",
"metricValue":17261,
"metricUnits":"bytes"
},
{
"metricDef":"success",
"metricType":"Counter",
"metricValue":100,
"metricUnits":"requests"
},
{
"metricDef":"dataPacketsSent",
"metricType":"Counter",
"metricValue":100,
"metricUnits":"packets"
},
{
"metricDef":"connectionCloseMs",
"metricType":"Timing",
"metricValueMap":{
"count":100,
"mean":8,
"min":3,
"p95":12,
"max":221,
"p99":43,
"sd":22,
"sum":886,
"var":497
},
"metricUnits":"ms"
},
{
"metricDef":"bytesReceived",
"metricType":"Counter",
"metricValue":9961,
"metricUnits":"bytes"
},
{
"metricDef":"dataPacketsReceived",
"metricType":"Counter",
"metricValue":200,
"metricUnits":"packets"
},
{
"metricDef":"firstReceivedMs",
"metricType":"Timing",
"metricValueMap":{
"count":100,
"mean":8,
"min":3,
"p95":11,
"max":217,
"p99":43,
"sd":21,
"sum":847,
"var":482
},
"metricUnits":"ms"
},
{
"metricDef":"bytesSent",
"metricType":"Counter",
"metricValue":7300,
"metricUnits":"bytes"
},
{
"metricDef":"dnsLookupMs",
"metricType":"Timing",
"metricValueMap":{
"count":100,
"mean":4,
"min":0,
"p95":8,
"max":212,
"p99":40,
"sd":21,
"sum":406,
"var":477
},
"metricUnits":"ms"
}
]
},
"script":{
"id":273,
"createdAt":"2016-02-18T19:00:46.862Z",
"updatedAt":"2016-03-09T08:10:44.945Z",
"name":"Default Scenario",
"length":97,
"hasInit":false,
"hasTeardown":false,
"scriptType":"Code"
}
}
Get execution summary
Returns summary metrics for this test execution.
GET /executions/:id/summary
Sample Response
{
"id":206866,
"createdAt":"2016-02-18T19:15:25.749Z",
"updatedAt":"2016-02-18T19:15:25.749Z",
"executionId":637,
"intervalStart":0,
"iterationsExecuted":100,
"count":100,
"concurrentClients":5,
"actualConcurrentClients":0,
"maxConcurrentClients":0,
"metrics":[
{
"metricDef":"httpResponseCode",
"metricType":"Histogram",
"metricValueMap":{
"200":100
}
},
{
"metricDef":"connectionOpenMs",
"metricType":"Timing",
"metricValueMap":{
"count":100,
"mean":5,
"min":1,
"p95":9,
"max":214,
"p99":41,
"sd":21,
"sum":527,
"var":481
},
"metricUnits":"ms"
},
{
"metricDef":"bandwidth",
"metricType":"Counter",
"metricValue":17261,
"metricUnits":"bytes"
},
{
"metricDef":"success",
"metricType":"Counter",
"metricValue":100,
"metricUnits":"requests"
},
{
"metricDef":"dataPacketsSent",
"metricType":"Counter",
"metricValue":100,
"metricUnits":"packets"
},
{
"metricDef":"connectionCloseMs",
"metricType":"Timing",
"metricValueMap":{
"count":100,
"mean":8,
"min":3,
"p95":12,
"max":221,
"p99":43,
"sd":22,
"sum":886,
"var":497
},
"metricUnits":"ms"
},
{
"metricDef":"bytesReceived",
"metricType":"Counter",
"metricValue":9961,
"metricUnits":"bytes"
},
{
"metricDef":"dataPacketsReceived",
"metricType":"Counter",
"metricValue":200,
"metricUnits":"packets"
},
{
"metricDef":"firstReceivedMs",
"metricType":"Timing",
"metricValueMap":{
"count":100,
"mean":8,
"min":3,
"p95":11,
"max":217,
"p99":43,
"sd":21,
"sum":847,
"var":482
},
"metricUnits":"ms"
},
{
"metricDef":"bytesSent",
"metricType":"Counter",
"metricValue":7300,
"metricUnits":"bytes"
},
{
"metricDef":"dnsLookupMs",
"metricType":"Timing",
"metricValueMap":{
"count":100,
"mean":4,
"min":0,
"p95":8,
"max":212,
"p99":40,
"sd":21,
"sum":406,
"var":477
},
"metricUnits":"ms"
}
]
}
Get execution metric
Returns aggregated metric details for a single metric. Works both during live test execution as well as after the test is finished running.
GET /executions/:id/metric?ns=[namespace]&name=[metric_name]
Sample Response
{
"namespace": "Testable",
"metricDef":"firstReceivedMs",
"metricType":"Timing",
"metricValueMap":{
"count":100,
"mean":8,
"min":3,
"p95":11,
"max":217,
"p99":43,
"sd":21,
"sum":847,
"var":482
},
"metricUnits":"ms"
}
Download execution results
Download the raw results, unaggregated for this execution. Can also be downloaded for one region where the test executed or for one resource. Note that the regionId
can be found in the execution details at regions[i].id
. Omit the .csv
at the end to get the results in JSON format.
GET /executions/:execution/results[.csv]
GET /executions/:execution/regions/:regionId/results[.csv]
GET /executions/:execution/resources/:resource/results[.csv]
GET /executions/:execution/regions/:regionId/resources/:resource/results[.csv]
Download assertion results
Download all assertions captured during this test. Includes any error messages and enough information to correlate the assertion with a particular virtual user, iteration, etc. Can also be downloaded for one region where the test executed or overall, either all assertions or just the ones with a failure. Note that the regionId
can be found in the execution details at regions[i].id
. Omit the .csv
at the end to get the results in JSON format.
GET /executions/:execution/suite-results[.csv]
GET /executions/:execution/regions/:regionId/suite-results[.csv]
GET /executions/:execution/failed-suite-results[.csv]
GET /executions/:execution/regions/:regionId/failed-suite-results[.csv]
Request Parameters
limit
: Number of assertions to return. Returns all by default or if set to -1.offset
: Offset within the result set to start the response. Defaults to 0. When combined withlimit
it is useful for paging through the assertion results.threadId
: Optional if you want to return results only for one user, pass the thread id.
Download assertion summary
Download a summary of all assertions captured during this test. Includes various counts like passed assertions, failed assertions, screenshots captured, etc. Can also be downloaded for one region where the test executed or overall. Note that the regionId
can be found in the execution details at regions[i].id
. Omit the .csv
at the end to get the results in JSON format.
GET /executions/:execution/suite-summaries[.csv]
GET /executions/:execution/regions/:regionId/suite-summaries[.csv]
Request Parameters
includeErrors
: Whether to include a row for each error encountered by assertion.
Download assertion errors summary
Download a summary of all errors that occurred while validating assertions in your test scenario. Can also be downloaded for one region where the test executed or overall. Note that the regionId
can be found in the execution details at regions[i].id
. Omit the .csv
at the end to get the results in JSON format.
GET /executions/:execution/suite-errors[.csv]
GET /executions/:execution/regions/:regionId/suite-errors[.csv]
Download traces
Download a ZIP file containing details on all traces captured during this test. For each trace there will be one file that contains the trace JSON and a second file that contains the packets captured for that trace. The file names correspond to the resource name of the trace with spaces and special characters replaced.
GET /executions/:execution/traces.zip
GET /executions/:execution/regions/:regionId/traces.zip
GET /executions/:execution/resources/:resource/traces.zip
GET /executions/:execution/regions/:regionId/resources/:resource/traces.zip
Get output file metadata
Download information including file name, size, concurrent user #, iteration #, timestamp, etc for all files captured during a test run.
GET /executions/:execution/output-files
Request Parameters
limit
: Number of output files to return. Returns all by default or if set to -1.offset
: Offset within the result set to start the response. Defaults to 0. When combined withlimit
it is useful for paging through the results.since
: Only return files wherecapturedAt >= since
. Time is specified as a unix epoch in milliseconds (e.g. 1457636340000). Optional.until
: Only return files wherecapturedAt < until
. Time is specified as a unix epoch in milliseconds (e.g. 1457636340000). Optional.contentType
: Only return files where the content type matches the specified value. Optional.agent
: Comma separated list of agent (i.e. test runner) ids to match. Only return files captured from the listed agents if specified. Optional.regions
: Comma separated list of execution region ids to match. Only return files captured in those regions if specified. Note that theregionId
can be found in the execution details atregions[i].id
. Optional.client
: Global user number (starts at 0) to match (globalConcurrentClientIndex
in the metadata). Only return files captured by the specified that user if specified. Optional.iteration
: Iteration number (starts at 0) to match. Only return files captured during the specified iteration if specified. Optional.failuresOnly
: If set to true only return files captured during test iterations that encountered assertion errors. Optional.
Response
[ {
"id":10566,
"createdAt":"2020-06-02T05:26:02.695Z",
"updatedAt":"2020-06-02T05:26:04.149Z",
"capturedAt":"2020-06-02T05:26:01.941Z",
"name":"test-0-2.png",
"type":"image/png",
"charset":"",
"size":30373,
"url":"https://api.testable.io/api/output-files/10566/download",
"viewUrl":"https://api.testable.io/api/output-files/10566/view",
"thumbnailUrl":"https://api.testable.io/api/output-files/10566/thumbnail",
"executionId":3383,
"executionRegionId":2599,
"regionId":1,
"locationId":1,
"chunkId":2695,
"chunkIndex":0,
"concurrentClientIndex":0,
"globalConcurrentClientIndex":0,
"regionalConcurrentClientIndex":0,
"iteration":0,
"diffPercent":0.38,
"diffDataStore":{
"id":8307,
"createdAt":"2020-06-02T05:23:26.990Z",
"updatedAt":"2020-06-02T05:23:26.990Z",
"name":"HomePage.png",
"type":"image/png",
"charset":"UTF-8",
"size":36906,
"url":"https://api.testable.io/api/data-stores/8307/download",
"viewUrl":"https://api.testable.io/api/data-stores/8307/view",
"thumbnailUrl":"https://api.testable.io/api/data-stores/8307/thumbnail",
"deleteUrl":"https://api.testable.io/api/data-stores/8307",
"deleteType":"DELETE",
"rows":[
],
"firstVersion":true
},
"expectDiff":true,
"suiteTestResultId":30122,
"threadId":"3383:aws-us-east-1:0",
"iterationId":"c4d880c4-ce73-4f7c-b773-e96361e0c954",
"isImage":true,
"isRelatedToFailure":true,
"agentId":3133,
"firstVersion":false
} ]
Download zip file of all output files
Download a zip file with all output files from this test run. Structured as follows unless you pass flat=true
or the test run is not a load test:
user-0
iteration-0
filexxx.txt
fileyyy.png
filezzz.jpg
iteration-1
...
user-1
...
GET /executions/:execution/output-files.zip
Request Parameters
flat
: Put all the files at the root of the zip file instead of grouping by user/iteration into folders. Defaults to false for load tests and true for monitors and functional tests.limit
: Number of output files to return. Returns all by default or if set to -1.offset
: Offset within the result set to start the response. Defaults to 0. When combined withlimit
it is useful for paging through the results.since
: Only return files wherecapturedAt >= since
. Time is specified as a unix epoch in milliseconds (e.g. 1457636340000). Optional.until
: Only return files wherecapturedAt < until
. Time is specified as a unix epoch in milliseconds (e.g. 1457636340000). Optional.contentType
: Only return files where the content type matches the specified value. Optional.agent
: Comma separated list of agent (i.e. test runner) ids to match. Only return files captured from the listed agents if specified. Optional.regions
: Comma separated list of execution region ids to match. Only return files captured in those regions if specified. Optional.client
: Global user number (starts at 0) to match (globalConcurrentClientIndex
in the metadata). Only return files captured by the specified that user if specified. Optional.iteration
: Iteration number (starts at 0) to match. Only return files captured during the specified iteration if specified. Optional.failuresOnly
: If set to true only return files captured during test iterations that encountered assertion errors. Optional.
Download file by name
Download an output file from your test results by file name. Note that if your test results include multiple files with the same name, the first match will be returned. You can pass the file ID as the fileName
parameter to choose a specific match if there are multiple with the same name. File IDs can be found using the GET /executions/:id/output-files
endpoint.
GET /executions/:id/output-files/:fileName/download
Export to PDF
Export the aggregated results to a PDF in the same format as you see via the website.
Request Parameters
dashboardId
: The specific view to use when exporting the results. If you are viewing the results via the website use the “View” dropdown at the top to switch between views. The URL in your browser will update with thedashboardId
that corresponds to the selected view. If not specified the default view for the test results is used.tz
: The timezone to use when displaying times in the PDF. Defaults to UTC. Use any valid Chrome value (e.g. America/Chicago).enc
: By default this endpoint will initiate a file download. If you passenc=base64
it will instead return the file base64 encoded.
GET /executions/:execution/results.pdf
Download assertions
Download the raw assertion results, unaggregated for this execution. Can also be downloaded for one region where the test executed. Note that the regionId
can be found in the execution details at regions[i].id
. Omit the .csv
at the end to get the results in JSON format.
GET /executions/:execution/suite-results[.csv]
GET /executions/:execution/regions/:regionId/suite-results[.csv]
Get execution intervals
Results are aggregated in 10 second buckets during execution. Can also be retrieved for one region where the test executed. Note that the regionId
can be found in the execution details at regions[i].id
.
GET /executions/:execution/intervals
GET /executions/:execution/regions/:regionId/intervals
Request Parameters
numIntervals
: Only return the latestnumIntervals
intervals based on timestamp. Optional.since
: Only return intervals whereintervalStart >= since
. Time is specified as a unix epoch in milliseconds (e.g. 1457636340000). Optional.
Response
[
{
"id":257947,
"createdAt":"2016-03-10T19:00:56.050Z",
"updatedAt":"2016-03-10T19:00:56.050Z",
"executionId":725,
"intervalStart":1457636450000,
"iterationsExecuted":3,
"count":6,
"concurrentClients":0,
"actualConcurrentClients":0,
"maxConcurrentClients":2,
"metrics":[
{
"metricDef":"httpResponseCode",
"metricType":"Histogram",
"metricValueMap":{
"200":3
}
},
{
"metricDef":"httpMethod",
"metricType":"Histogram",
"metricValueMap":{
"GET":3
}
},
{
"metricDef":"connectionOpenMs",
"metricType":"Timing",
"metricValueMap":{
"count":3,
"mean":1,
"min":1,
"p95":2,
"max":2,
"p99":2,
"sd":0,
"p50":2,
"sum":5,
"var":0
},
"metricUnits":"ms"
},
{
"metricDef":"bandwidth",
"metricType":"Counter",
"metricValue":521,
"metricUnits":"bytes"
},
{
"metricDef":"success",
"metricType":"Counter",
"metricValue":6,
"metricUnits":"requests"
},
{
"metricDef":"dataPacketsSent",
"metricType":"Counter",
"metricValue":3,
"metricUnits":"packets"
},
{
"metricDef":"connectionCloseMs",
"metricType":"Timing",
"metricValueMap":{
"count":3,
"mean":5,
"min":4,
"p95":7,
"max":7,
"p99":7,
"sd":1,
"p50":6,
"sum":17,
"var":2
},
"metricUnits":"ms"
},
{
"metricDef":"tickLatencyMs",
"metricType":"Timing",
"metricValueMap":{
"count":3,
"mean":123,
"min":123,
"p95":123,
"max":123,
"p99":123,
"sd":0,
"p50":123,
"sum":369,
"var":0
},
"metricUnits":"ms"
},
{
"metricDef":"bytesReceived",
"metricType":"Counter",
"metricValue":302,
"metricUnits":"bytes"
},
{
"metricDef":"dataPacketsReceived",
"metricType":"Counter",
"metricValue":6,
"metricUnits":"packets"
},
{
"metricDef":"firstReceivedMs",
"metricType":"Timing",
"metricValueMap":{
"count":3,
"mean":5,
"min":4,
"p95":7,
"max":7,
"p99":7,
"sd":1,
"p50":6,
"sum":17,
"var":2
},
"metricUnits":"ms"
},
{
"metricDef":"bytesSent",
"metricType":"Counter",
"metricValue":219,
"metricUnits":"bytes"
},
{
"metricDef":"dnsLookupMs",
"metricType":"Timing",
"metricValueMap":{
"count":3,
"mean":0,
"min":0,
"p95":1,
"max":1,
"p99":1,
"sd":0,
"p50":0,
"sum":1,
"var":0
},
"metricUnits":"ms"
}
]
}
]
Get execution results by resource
Results are aggregated by resource (e.g. GET http://myserver.com/somethingTested
). The resource name is specified during test execution either automatically by Testable or by you when capturing a custom metric.
Can also be retrieved for one region where the test executed. Note that the regionId
can be found in the execution details at regions[i].id
.
GET /executions/:execution/resources
GET /executions/:execution/regions/:regionId/resources
Request Parameters
intervals
: Whether or not to include the by interval results for each resource. Defaults to false. Note that this will return a huge amount of data and it is recommended you retrieve interval data for either the entire execution or for one specific resource.
Response
[
{
"resource":{
"id":239,
"createdAt":"2016-01-31T23:57:52.628Z",
"updatedAt":"2016-01-31T23:57:52.629Z",
"resource":"GET http://sample.testable.io/stocks/IBM/stocks/IBM"
},
"summary":{
"id":257943,
"createdAt":"2016-03-10T19:00:56.030Z",
"updatedAt":"2016-03-10T19:00:56.030Z",
"executionId":725,
"resourceId":239,
"intervalStart":0,
"iterationsExecuted":0,
"count":100,
"concurrentClients":0,
"actualConcurrentClients":0,
"maxConcurrentClients":0,
"metrics":[
{
"metricDef":"httpResponseCode",
"metricType":"Histogram",
"metricValueMap":{
"200":100
}
},
{
"metricDef":"httpMethod",
"metricType":"Histogram",
"metricValueMap":{
"GET":100
}
},
{
"metricDef":"connectionOpenMs",
"metricType":"Timing",
"metricValueMap":{
"count":100,
"mean":3,
"min":1,
"p95":17,
"max":33,
"p99":28,
"sd":5,
"p50":2,
"sum":311,
"var":27
},
"metricUnits":"ms"
},
{
"metricDef":"bandwidth",
"metricType":"Counter",
"metricValue":17264,
"metricUnits":"bytes"
},
{
"metricDef":"success",
"metricType":"Counter",
"metricValue":100,
"metricUnits":"requests"
},
{
"metricDef":"dataPacketsSent",
"metricType":"Counter",
"metricValue":100,
"metricUnits":"packets"
},
{
"metricDef":"connectionCloseMs",
"metricType":"Timing",
"metricValueMap":{
"count":100,
"mean":6,
"min":3,
"p95":19,
"max":41,
"p99":32,
"sd":5,
"p50":6,
"sum":689,
"var":31
},
"metricUnits":"ms"
},
{
"metricDef":"bytesReceived",
"metricType":"Counter",
"metricValue":9964,
"metricUnits":"bytes"
},
{
"metricDef":"dataPacketsReceived",
"metricType":"Counter",
"metricValue":200,
"metricUnits":"packets"
},
{
"metricDef":"firstReceivedMs",
"metricType":"Timing",
"metricValueMap":{
"count":100,
"mean":6,
"min":3,
"p95":19,
"max":38,
"p99":31,
"sd":5,
"p50":5,
"sum":646,
"var":29
},
"metricUnits":"ms"
},
{
"metricDef":"bytesSent",
"metricType":"Counter",
"metricValue":7300,
"metricUnits":"bytes"
},
{
"metricDef":"dnsLookupMs",
"metricType":"Timing",
"metricValueMap":{
"count":100,
"mean":1,
"min":0,
"p95":16,
"max":31,
"p99":27,
"sd":5,
"p50":1,
"sum":193,
"var":27
},
"metricUnits":"ms"
}
]
}
}
]
Get execution results for a resource
Get results for one specific resource. See the get execution results by resource for more explanation on resources. Use that route to get the resource id.
Can also be retrieved for one region where the test executed. Note that the regionId
can be found in the execution details at regions[i].id
.
GET /executions/:execution/resources/:resourceId
GET /executions/:execution/regions/:regionId/resources/:resourceId
Request Parameters
intervals
: Whether or not to include the by interval results. Defaults to true.
Response
{
"resource":{
"id":239,
"createdAt":"2016-01-31T23:57:52.628Z",
"updatedAt":"2016-01-31T23:57:52.629Z",
"resource":"GET http://sample.testable.io/stocks/IBM/stocks/IBM"
},
"intervals": [],
"summary":{
"id":257943,
"createdAt":"2016-03-10T19:00:56.030Z",
"updatedAt":"2016-03-10T19:00:56.030Z",
"executionId":725,
"resourceId":239,
"intervalStart":0,
"iterationsExecuted":0,
"count":100,
"concurrentClients":0,
"actualConcurrentClients":0,
"maxConcurrentClients":0,
"metrics":[
{
"metricDef":"httpResponseCode",
"metricType":"Histogram",
"metricValueMap":{
"200":100
}
},
{
"metricDef":"httpMethod",
"metricType":"Histogram",
"metricValueMap":{
"GET":100
}
},
{
"metricDef":"connectionOpenMs",
"metricType":"Timing",
"metricValueMap":{
"count":100,
"mean":3,
"min":1,
"p95":17,
"max":33,
"p99":28,
"sd":5,
"p50":2,
"sum":311,
"var":27
},
"metricUnits":"ms"
},
{
"metricDef":"bandwidth",
"metricType":"Counter",
"metricValue":17264,
"metricUnits":"bytes"
},
{
"metricDef":"success",
"metricType":"Counter",
"metricValue":100,
"metricUnits":"requests"
},
{
"metricDef":"dataPacketsSent",
"metricType":"Counter",
"metricValue":100,
"metricUnits":"packets"
},
{
"metricDef":"connectionCloseMs",
"metricType":"Timing",
"metricValueMap":{
"count":100,
"mean":6,
"min":3,
"p95":19,
"max":41,
"p99":32,
"sd":5,
"p50":6,
"sum":689,
"var":31
},
"metricUnits":"ms"
},
{
"metricDef":"bytesReceived",
"metricType":"Counter",
"metricValue":9964,
"metricUnits":"bytes"
},
{
"metricDef":"dataPacketsReceived",
"metricType":"Counter",
"metricValue":200,
"metricUnits":"packets"
},
{
"metricDef":"firstReceivedMs",
"metricType":"Timing",
"metricValueMap":{
"count":100,
"mean":6,
"min":3,
"p95":19,
"max":38,
"p99":31,
"sd":5,
"p50":5,
"sum":646,
"var":29
},
"metricUnits":"ms"
},
{
"metricDef":"bytesSent",
"metricType":"Counter",
"metricValue":7300,
"metricUnits":"bytes"
},
{
"metricDef":"dnsLookupMs",
"metricType":"Timing",
"metricValueMap":{
"count":100,
"mean":1,
"min":0,
"p95":16,
"max":31,
"p99":27,
"sd":5,
"p50":1,
"sum":193,
"var":27
},
"metricUnits":"ms"
}
]
}
}
Get execution logs
Get all logging captured during execution either in JSON or text format.
GET /executions/:execution/logs[.txt]
Request Parameters
limit
: Number of log entries to return. Returns all by default or if set to -1.asc
: Whether to return the logs in ascending or descending order.true
(ascending) by default.systemOnly
: Whether to return only system log entries or all log entries including user generating ones.false
by default.
Response
JSON:
{
"size":8,
"logs":[
{
"id":38666,
"level":"Info",
"msg":"Execution started",
"length":17,
"isSystem":true,
"createdAt":"2016-03-10T18:57:37.069Z",
"updatedAt":"2016-03-10T18:57:37.091Z",
"loggedAt":"2016-03-10T18:57:37.069Z"
},
{
"id":38667,
"level":"Info",
"msg":"Allocating to agents in AWS N. Virginia",
"length":39,
"isSystem":true,
"createdAt":"2016-03-10T18:57:37.100Z",
"updatedAt":"2016-03-10T18:57:37.102Z",
"loggedAt":"2016-03-10T18:57:37.100Z"
},
{
"id":38668,
"level":"Info",
"msg":"Allocated to agents in AWS N. Virginia. Testing is underway",
"length":59,
"isSystem":true,
"createdAt":"2016-03-10T18:57:37.160Z",
"updatedAt":"2016-03-10T18:57:37.162Z",
"loggedAt":"2016-03-10T18:57:37.160Z"
},
{
"id":38669,
"level":"Info",
"msg":"Finished execution in AWS N. Virginia",
"length":37,
"isSystem":true,
"createdAt":"2016-03-10T19:00:55.821Z",
"updatedAt":"2016-03-10T19:00:55.823Z",
"loggedAt":"2016-03-10T19:00:55.821Z"
},
{
"id":38670,
"level":"Info",
"msg":"Finished execution globally, processing 200 results",
"length":51,
"isSystem":true,
"createdAt":"2016-03-10T19:00:55.846Z",
"updatedAt":"2016-03-10T19:00:55.848Z",
"loggedAt":"2016-03-10T19:00:55.846Z"
},
{
"id":38671,
"level":"Info",
"msg":"Analyzing and aggregating results",
"length":33,
"isSystem":true,
"createdAt":"2016-03-10T19:00:55.980Z",
"updatedAt":"2016-03-10T19:00:55.984Z",
"loggedAt":"2016-03-10T19:00:55.980Z"
},
{
"id":38672,
"level":"Info",
"msg":"Finished processing results, cleaning up resources",
"length":50,
"isSystem":true,
"createdAt":"2016-03-10T19:00:56.805Z",
"updatedAt":"2016-03-10T19:00:56.807Z",
"loggedAt":"2016-03-10T19:00:56.805Z"
},
{
"id":38673,
"level":"Info",
"msg":"Test completed",
"length":14,
"isSystem":true,
"createdAt":"2016-03-10T19:00:56.809Z",
"updatedAt":"2016-03-10T19:00:56.813Z",
"loggedAt":"2016-03-10T19:00:56.809Z"
}
]
}
Text:
2016-03-10T18:57:37.069Z INFO Execution started
2016-03-10T18:57:37.100Z INFO Allocating to agents in AWS N. Virginia
2016-03-10T18:57:37.160Z INFO Allocated to agents in AWS N. Virginia. Testing is underway
2016-03-10T19:00:55.821Z INFO Finished execution in AWS N. Virginia
2016-03-10T19:00:55.846Z INFO Finished execution globally, processing 200 results
2016-03-10T19:00:55.980Z INFO Analyzing and aggregating results
2016-03-10T19:00:56.805Z INFO Finished processing results, cleaning up resources
2016-03-10T19:00:56.809Z INFO Test completed
Get execution traces
Get all connection traces captured as part of this execution. Traces allow you to see exactly what happened during a connection including metrics, data sent, and data received.
Can also be retrieved for one region where the test executed. Note that the regionId
can be found in the execution details at regions[i].id
.
GET /executions/:execution/traces
GET /executions/:execution/regions/:regionId/traces
Request Parameters
limit
: Number of traces to return. Returns all by default or if set to -1.offset
: Offset within the results to start the response. Defaults to 0. When combined withlimit
it is useful for paging through the results.
Response
[
{
"id":430,
"length":219,
"createdAt":"2016-03-19T19:42:36.889Z",
"updatedAt":"2016-03-19T19:42:36.889Z",
"traceAt":"2016-03-19T19:42:32.466Z",
"status":"301",
"regionId":857,
"resource":"GET http://google.com/"
},
{
"id":431,
"length":0,
"createdAt":"2016-03-19T19:42:37.471Z",
"updatedAt":"2016-03-19T19:42:37.471Z",
"traceAt":"2016-03-19T19:42:32.477Z",
"status":"204",
"regionId":857,
"resource":"GET http://www.google.com/client_204?&atyp=i&biw=400&bih=300&ei=92LgVrP-Bszb-QHPlZXoCA"
}
]
Get execution trace
Get all details of a trace captured as part of this execution. Traces allow you to see exactly what happened during a connection including metrics, data sent, and data received.
Packet data is base64 encoded. In Javascript it can be decoded using atob(data)
.
GET /execution-traces/:id
Request Parameters
up
: Whether or not to return the parent entity, the execution region in this case. Defaults to true.down
: Whether or not to return the child entities, the trace packets in this case. Defaults to true.includeResult
: Whether or not to return the result metrics captured during this connection. Defaults to true.
Response
{
"id":430,
"length":219,
"createdAt":"2016-03-19T19:42:36.889Z",
"updatedAt":"2016-03-19T19:42:36.889Z",
"traceAt":"2016-03-19T19:42:32.466Z",
"status":"301",
"regionId":857,
"resource":"GET http://google.com/",
"result":{
"id":1455678,
"createdAt":"2016-03-19T19:42:36.763Z",
"updatedAt":"2016-03-19T19:42:36.767Z",
"executionId":809,
"executionRegionId":857,
"chunkId":813,
"connectionId":"5ab973d0-e828-4049-a03b-56017710c5b8",
"agentId":"3cd00a3a-0387-42a0-be1d-85d6899008d1",
"threadId":"86ebef43-11a2-400e-8f60-48729000e39f",
"timestamp":"2016-03-19T19:42:32.466Z",
"resource":{
"id":1,
"createdAt":"2015-08-11T17:06:34.780Z",
"updatedAt":"2015-08-11T17:06:34.781Z",
"resource":"GET http://google.com/"
},
"metrics":[
{
"metricDef":"dnsLookupMs",
"metricType":"Timing",
"metricValue":19,
"metricUnits":"ms"
},
{
"metricDef":"connectionOpenMs",
"metricType":"Timing",
"metricValue":22,
"metricUnits":"ms"
},
{
"metricDef":"firstReceivedMs",
"metricType":"Timing",
"metricValue":47,
"metricUnits":"ms"
},
{
"metricDef":"httpMethod",
"metricType":"Histogram",
"metricKey":"GET",
"metricValue":1
},
{
"metricDef":"dataPacketsReceived",
"metricType":"Counter",
"metricValue":2,
"metricUnits":"packets"
},
{
"metricDef":"httpResponseCode",
"metricType":"Histogram",
"metricKey":"301",
"metricValue":1
},
{
"metricDef":"success",
"metricType":"Counter",
"metricValue":1,
"metricUnits":"requests"
},
{
"metricDef":"connectionCloseMs",
"metricType":"Timing",
"metricValue":50,
"metricUnits":"ms"
},
{
"metricDef":"bytesSent",
"metricType":"Counter",
"metricValue":300,
"metricUnits":"bytes"
},
{
"metricDef":"bytesReceived",
"metricType":"Counter",
"metricValue":540,
"metricUnits":"bytes"
},
{
"metricDef":"bandwidth",
"metricType":"Counter",
"metricValue":840,
"metricUnits":"bytes"
}
]
},
"packets":[
{
"id":2370,
"length":0,
"packetType":"DnsLookup",
"dataLength":0,
"createdAt":"2016-03-19T19:42:36.896Z",
"updatedAt":"2016-03-19T19:42:36.896Z",
"packetAt":"2016-03-19T19:42:32.484Z",
"lastPacketAt":"2016-03-19T19:42:32.484Z"
},
{
"id":2371,
"length":0,
"packetType":"ConnectionOpened",
"dataLength":0,
"createdAt":"2016-03-19T19:42:36.896Z",
"updatedAt":"2016-03-19T19:42:36.896Z",
"packetAt":"2016-03-19T19:42:32.488Z",
"lastPacketAt":"2016-03-19T19:42:32.488Z"
},
{
"id":2372,
"length":0,
"packetType":"DataSent",
"headers":"{\"User-Agent\":\"Mozilla/5.0 (Unknown; Linux x86_64) AppleWebKit/534.34 (KHTML, like Gecko) PhantomJS/1.9.7 Safari/534.34\",\"Accept\":\"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\",\"Connection\":\"Keep-Alive\",\"Accept-Encoding\":\"gzip\",\"Accept-Language\":\"en-US,*\",\"Host\":\"google.com\"}",
"dataLength":0,
"createdAt":"2016-03-19T19:42:36.896Z",
"updatedAt":"2016-03-19T19:42:36.896Z",
"packetAt":"2016-03-19T19:42:32.488Z",
"lastPacketAt":"2016-03-19T19:42:32.488Z"
},
{
"id":2373,
"length":219,
"packetType":"DataReceived",
"headers":"{\"location\":\"http://www.google.com/\",\"content-type\":\"text/html; charset=UTF-8\",\"date\":\"Sat, 19 Mar 2016 19:42:32 GMT\",\"expires\":\"Mon, 18 Apr 2016 19:42:32 GMT\",\"cache-control\":\"public, max-age=2592000\",\"server\":\"gws\",\"content-length\":\"219\",\"x-xss-protection\":\"1; mode=block\",\"x-frame-options\":\"SAMEORIGIN\"}",
"data":"PEhUTUw+PEhFQUQ+PG1ldGEgaHR0cC1lcXVpdj0iY29udGVudC10eXBlIiBjb250ZW50PSJ0ZXh0L2h0bWw7Y2hhcnNldD11dGYtOCI+CjxUSVRMRT4zMDEgTW92ZWQ8L1RJVExFPjwvSEVBRD48Qk9EWT4KPEgxPjMwMSBNb3ZlZDwvSDE+ClRoZSBkb2N1bWVudCBoYXMgbW92ZWQKPEEgSFJFRj0iaHR0cDovL3d3dy5nb29nbGUuY29tLyI+aGVyZTwvQT4uDQo8L0JPRFk+PC9IVE1MPg0K",
"dataLength":219,
"createdAt":"2016-03-19T19:42:36.896Z",
"updatedAt":"2016-03-19T19:42:36.896Z",
"packetAt":"2016-03-19T19:42:32.513Z",
"lastPacketAt":"2016-03-19T19:42:32.515Z"
},
{
"id":2374,
"length":0,
"packetType":"ConnectionClosed",
"dataLength":0,
"createdAt":"2016-03-19T19:42:36.896Z",
"updatedAt":"2016-03-19T19:42:36.896Z",
"packetAt":"2016-03-19T19:42:32.517Z",
"lastPacketAt":"2016-03-19T19:42:32.517Z"
}
]
}
Start execution
Starts a new execution of the test configuration.
POST /test-confs/:confId/executions
Request Parameters
None. All configuration parameters are captured when creating the test configuration.
Response
{
"id":735,
"createdAt":"2016-03-12T05:15:38.160Z",
"updatedAt":"2016-03-12T05:15:38.179Z",
"running":false,
"allocated":false,
"cancelledByUser":false,
"completed":false,
"summaryCalculated":false,
"limitBreached":false,
"iterations":20,
"iterationSleepSecs":10,
"rampupSecs":0,
"regions":[
{
"id":773,
"createdAt":"2016-03-12T05:15:38.192Z",
"updatedAt":"2016-03-12T05:15:38.192Z",
"running":false,
"completed":false,
"concurrentClients":5,
"actualConcurrentClients":5,
"expectedResultCount":100,
"region":{
"id":1,
"createdAt":"2015-08-11T17:03:34.761Z",
"updatedAt":"2015-08-11T17:03:34.761Z",
"name":"aws-us-east-1",
"public":true,
"latitude":39.0436,
"longitude":-77.4878,
"description":"AWS N. Virginia",
"active":true
}
}
],
"concurrentClients":5,
"actualConcurrentClients":5,
"expectedResultCount":100,
"scriptParams":"{}",
"script":{
"id":273,
"createdAt":"2016-02-18T19:00:46.862Z",
"updatedAt":"2016-03-09T08:10:44.945Z",
"name":"Default Scenario",
"length":97,
"hasInit":false,
"hasTeardown":false,
"scriptType":"Code"
}
}
Check if execution exists
HEAD /executions/:id
Response
HTTP Status: 200 OK or 404 NOT FOUND
Start execution
Request to start the execution. Note that creating and starting an execution are two steps.
PUT /executions/:id/start
Request Parameters
None
Response
{
"updatedAt":"2016-03-12T05:15:41.830Z",
"iterationSleepSecs":10,
"allocated":false,
"limitBreached":false,
"running":true,
"summaryCalculated":false,
"rampupSecs":0,
"actualConcurrentClients":5,
"regions":[
{
"updatedAt":"2016-03-12T05:15:38.192Z",
"running":false,
"actualConcurrentClients":5,
"id":773,
"completed":false,
"createdAt":"2016-03-12T05:15:38.192Z",
"region":{
"name":"aws-us-east-1",
"updatedAt":"2015-08-11T17:03:34.761Z",
"latitude":39.0436,
"description":"AWS N. Virginia",
"public":true,
"longitude":-77.4878,
"id":1,
"createdAt":"2015-08-11T17:03:34.761Z",
"active":true
},
"concurrentClients":5,
"expectedResultCount":100
}
],
"id":735,
"completed":false,
"scriptParams":"{}",
"cancelledByUser":false,
"createdAt":"2016-03-12T05:15:38.160Z",
"startedAt":"2016-03-12T05:15:41.829Z",
"script":{
"name":"Default Scenario",
"updatedAt":"2016-03-09T08:10:44.945Z",
"hasTeardown":false,
"scriptType":"Code",
"id":273,
"createdAt":"2016-02-18T19:00:46.862Z",
"length":97,
"hasInit":false
},
"iterations":20,
"concurrentClients":5,
"expectedResultCount":100
}
Manual start
Tests that were started with manual_start=true will simply sit and wait after the test runners are allocated and initialized. This route tells Testable to start executing the test across the test runners. This can be useful with on-demand test runners since we don’t know exactly how long AWS EC2 instances take to spin up. Start the test with manual_start=true well in advance of your desired test run time and when you trigger the manual start it will immediately start generating load.
Stop execution
Request to stop the execution before it completes. Note that once the execution is stopped it will take a little while before it has completed: true
status due to cleanup, result aggregation, etc.
PUT /executions/:id/stop
Request Parameters
None
Response
{
"updatedAt":"2016-03-12T05:15:41.830Z",
"iterationSleepSecs":10,
"allocated":false,
"limitBreached":false,
"running":false,
"summaryCalculated":false,
"rampupSecs":0,
"actualConcurrentClients":5,
"regions":[
{
"updatedAt":"2016-03-12T05:15:38.192Z",
"running":false,
"actualConcurrentClients":5,
"id":773,
"completed":false,
"createdAt":"2016-03-12T05:15:38.192Z",
"region":{
"name":"aws-us-east-1",
"updatedAt":"2015-08-11T17:03:34.761Z",
"latitude":39.0436,
"description":"AWS N. Virginia",
"public":true,
"longitude":-77.4878,
"id":1,
"createdAt":"2015-08-11T17:03:34.761Z",
"active":true
},
"concurrentClients":5,
"expectedResultCount":100
}
],
"id":735,
"completed":false,
"scriptParams":"{}",
"cancelledByUser":true,
"createdAt":"2016-03-12T05:15:38.160Z",
"startedAt":"2016-03-12T05:15:41.829Z",
"finishedAt":"2016-03-12T05:15:42.736Z",
"script":{
"name":"Default Scenario",
"updatedAt":"2016-03-09T08:10:44.945Z",
"hasTeardown":false,
"scriptType":"Code",
"id":273,
"createdAt":"2016-02-18T19:00:46.862Z",
"length":97,
"hasInit":false
},
"iterations":20,
"concurrentClients":5,
"expectedResultCount":100
}
Delete execution
Delete this execution. Only works once the execution is completed. If you want to delete before it completes, please stop first and wait until the completed
field is true
.
DELETE /executions/:id
Response
The deleted execution
{
"updatedAt":"2016-03-12T05:15:41.830Z",
"iterationSleepSecs":10,
"allocated":true,
"limitBreached":false,
"running":false,
"summaryCalculated":true,
"rampupSecs":0,
"actualConcurrentClients":5,
"regions":[
{
"updatedAt":"2016-03-12T05:15:38.192Z",
"running":false,
"actualConcurrentClients":5,
"id":773,
"completed":true,
"createdAt":"2016-03-12T05:15:38.192Z",
"region":{
"name":"aws-us-east-1",
"updatedAt":"2015-08-11T17:03:34.761Z",
"latitude":39.0436,
"description":"AWS N. Virginia",
"public":true,
"longitude":-77.4878,
"id":1,
"createdAt":"2015-08-11T17:03:34.761Z",
"active":true
},
"concurrentClients":5,
"expectedResultCount":100
}
],
"id":735,
"completed":true,
"scriptParams":"{}",
"cancelledByUser":true,
"createdAt":"2016-03-12T05:15:38.160Z",
"startedAt":"2016-03-12T05:15:41.829Z",
"finishedAt":"2016-03-12T05:15:42.736Z",
"script":{
"name":"Default Scenario",
"updatedAt":"2016-03-09T08:10:44.945Z",
"hasTeardown":false,
"scriptType":"Code",
"id":273,
"createdAt":"2016-02-18T19:00:46.862Z",
"length":97,
"hasInit":false
},
"iterations":20,
"concurrentClients":5,
"expectedResultCount":100
}
Extend execution
Changes the duration or number of iterations this test will execute for. Propogates live to all test runners currently running the test. If the new duration/iterations has already passed the test will complete once the currently executing iteration is completed.
This only works for scenario types where Testable manages the concurrency (i.e. not JMeter or Gatling).
POST /executions/:executionId/live-extension
Request Body
For tests that are configured to run for a duration (in seconds):
{
"newDuration": 360
}
For tests that are configured to run for a certain number of iterations:
{
"newIterations": 5
}
Response
200 OK status
Add users/engines
Adds additional virtual users or engines (for JMeter/Gatling/Locust) on the fly while the test is running. If the test is utilizing per-test runners, additional runners may be provisioned first.
POST /executions/:executionId/add-users
Request Body
{
"users": 5,
"executionRegion": 12345
}
users
: Number of virtual users or engines to add. For JMeter/Gatling/Locust this number corresponds to the number of engines to add.executionRegion
: Which region to add the users. The execution region id can be found using theGET /executions/:id
API atregions[*].id
in the response.
Response
200 OK status
Send event
Send a live event to all virtual users while the test is running
POST /executions/:executionId/send-event
Request Body
The name of the event (required) and any contents to pass along (optional).
{
"name": "load-url",
"contents": "https://www.amazon.com"
}
Response
200 OK status
Share results
Share this test result via a public share link. Anyone who has the link can see the results without logging in.
POST /executions/:executionId/share
Request Body
Optionally specify the view id and/or timezone that you want people to see the results with when accessing the share link
{
"dashboardId": 0,
"tz": "America/Chicago"
}
Response
{
"id":1773,
"createdAt":"2022-12-12T22:52:08.400Z",
"updatedAt":"2022-12-12T22:52:08.400Z",
"name":"rprimspglm",
"targetType":"GetLoadTestExecution",
"targetId":15450,
"description":"{\"dashboardId\":0,\"tz\":\"America/Chicago\"}",
"active":true,
"organization":{
"id":2
},
"firstVersion":true
}
The name
property can then be used as the public token via https://a.testable.io/p/:name
to access the test results. Use https://a.testable.io/p/:name/results.pdf
to download the results as a PDF.