Monitor Success Rate Using Our API

In this example we do the following:

  1. Start a test execution
  2. Wait for it to finish. If the success rate while running dips below 50% after we have at least 5 results, stop the test immediately.
  3. Save all results to a CSV.

It assumes you have already created a test configuration and scenario via the API or website first and have the trigger URL for the configuration available, new configuration guide, new scenario guide, configuration API, and scenario API for more details.

A trigger URL is required in the first API call below.


echo "[$(date)] Start a new execution for existing trigger"
execution_id=$(curl -X POST --silent | jq -r ".id")

# This next part keeps checking the success rate once we have 5 results until the test is done.
# If it dips below 50% it stops execution.

echo "[$(date)] Waiting for execution to complete (view online at$execution_id)"
echo "[$(date)] Will stop execution if the success rate dips below 50%"
while : ; do
  echo -n "."
  sleep 5
  details=$(curl -H "X-Testable-Key:$API_KEY" --silent$execution_id)
  running=$(echo "$details" | jq -r ".running")
  if [[ $running = "true" ]]; then
    count=$(echo "$details" | jq -r ".summary.count")
    success=$(echo "$details" | jq -r '.summary.metrics | .[] | select(.metricDef=="success") | .metricValue')
    if [[ $count > 5 ]]; then
      pct=$(( $success / $count * 100 ))
      if [[ $pct < 50 ]]; then
        echo "[$(date)] Success rate down to $pct %, stopping execution"
        curl -H "X-Testable-Key:$API_KEY" -X PATCH --silent$execution_id/stop &>/dev/null
        echo "[$(date)] Stopped execution"
  [[ $(echo "$details" | jq -r ".completed") = "false" ]] || break

epoch=$(date +"%s")
echo "[$(date)] Storing CSV results at results-$epoch.csv"
curl -H "X-Testable-Key:$API_KEY" --silent$execution_id/results.csv > results-$epoch.csv