Monitor Success Rate and Response Time Using Our API

In this example we do the following:

  1. Run a test with success rate and response time KPIs
  2. Wait for it to finish
  3. Print the success/failure status and the observed success rate and median response time
  4. Save all results to a CSV.

Running a test with KPIs

Use the simple API to start a test that runs 5 concurrent users of our Node.js script. Set success rate > 95% and median response time < 300ms as our KPIs. Testable will automatically evaluate them at the end of the test when determining the success/faiure status.

curl -sS \
  -F "code=@/Users/Avi/Downloads/test2.js" \
  -F "concurrent_users_per_region=5" \
  -F "duration_mins=1" \
  -F "rampup_mins=0" \
  -F "conf_testrunners[0].regions=aws-us-east-1" \
  -F "kpis[0].expr=Outcome[success] > 95%" \
  -F "kpis[1].expr=Response Time[Median] < 300ms" \
  -F "testcase_name=Api Test" \
  https://api.testable.io/start?key=$TESTABLE_KEY

testable-kpi-monitor.sh

Use our example testable-kpi-monitor.sh script to monitor the test execution, print the success/failure status + success rate + median response time, and save the raw results to a CSV file.

Putting it all together:

./testable-kpi-monitor.sh $(curl -sS \
  -F "code=@/Users/Avi/Downloads/test2.js" \
  -F "concurrent_users_per_region=5" \
  -F "duration_mins=1" \
  -F "rampup_mins=0" \
  -F "conf_testrunners[0].regions=aws-us-east-1" \
  -F "kpis[0].expr=Outcome[success] > 95%" \
  -F "kpis[1].expr=Response Time[Median] < 300ms" \
  -F "testcase_name=Api Test" \
  https://api.testable.io/start?key=$TESTABLE_KEY)

Example output:

[Wed Dec 19 12:23:08 CST 2018] Waiting for execution to complete (view online at https://a.testable.io/results/xxx)
.................
[Wed Dec 19 12:24:59 CST 2018] Test SUCCESS
[Wed Dec 19 12:24:59 CST 2018] Median Response Time: 30ms
[Wed Dec 19 12:24:59 CST 2018] Success Rate: 100%
[Wed Dec 19 12:24:59 CST 2018] Storing CSV results at results-1545243899.csv