# qa.pro from CI

Create an API key in Settings → API keys (Team plan and up) and store it as a secret. The workflow below tests every successful preview
deployment. qa.pro takes preview URLs under a domain you verified (a verified apex covers
`preview-123.shop.example`); any other host waits until its domain is verified, and the run's `waiting` field says so.

```yaml
# .github/workflows/qapro.yml
name: qa.pro
on: [deployment_status]
jobs:
  check:
    if: github.event.deployment_status.state == 'success'
    runs-on: ubuntu-latest
    steps:
      - name: Test the preview URL
        env:
          QAPRO_URL: https://qa.pro
          QAPRO_KEY: ${{ secrets.QAPRO_KEY }}
          TARGET: ${{ github.event.deployment_status.target_url }}
        run: |
          set -u
          run=$(curl -sS --fail-with-body -X POST "$QAPRO_URL/api/v1/runs" \
                -H "Authorization: Bearer $QAPRO_KEY" \
                -H "Content-Type: application/json" \
                -d "{\"urls\": [\"$TARGET\"]}") \
            || { echo "::error::could not start the run: $run"; exit 1; }
          # a run may be split in two when some addresses wait for their domain's proof:
          # poll every id the API returned
          ids=$(echo "$run" | python3 -c '
          import json, sys
          d = json.load(sys.stdin)
          print(" ".join(str(i) for i in [d["id"]] + d.get("also_queued", [])))')
          fail=0
          for id in $ids; do
            echo "run $QAPRO_URL/runs/$id"
            status=queued; st=""
            for i in $(seq 1 90); do
              sleep 20
              # a transient 502 must not fail the job
              st=$(curl -sS --fail-with-body "$QAPRO_URL/api/v1/runs/$id" \
                   -H "Authorization: Bearer $QAPRO_KEY") \
                || { echo "poll: $st"; sleep 20; continue; }
              status=$(echo "$st" |
                python3 -c 'import json,sys; print(json.load(sys.stdin)["status"])')
              if [ "$status" = done ] || [ "$status" = failed ] ||
                 [ "$status" = cancelled ]; then break; fi
            done
            # nothing finished it in 30 minutes: say why
            if [ "$status" = queued ] || [ "$status" = running ]; then
              waiting=$(echo "$st" | python3 -c '
          import json, sys
          print(json.load(sys.stdin).get("waiting") or "no browser was free")')
              echo "::error::run $id: $waiting"; fail=1; continue
            fi
            echo "$st" | python3 -c '
          import json, sys
          d = json.load(sys.stdin); s = d["summary"]; r = s["failing_sites"]
          print("run %d: %d passed · %d failed · %d not run; %d of %d browsers reported"
                % (d["id"], s["pass"], s.get("fail", 0), s.get("not_checked", 0),
                   s["results"], s["expected"]))
          # needs a look: the check stopped and nothing on the page says whose fault;
          # watch its film on the report
          if s.get("look"):
              print("run %d: %d %s a look; that is not a failure: watch what happened on %s"
                    % (d["id"], s["look"], "needs" if s["look"] == 1 else "need",
                       d["url"]))
          # a look is not a failure
          ok = (d["status"] == "done" and r == 0 and s["ended"] + s.get("look", 0) > 0
                and s["results"] >= s["expected"])
          sys.exit(0 if ok else 1)' || fail=1
          done
          exit $fail
```

A check that needs a look (it stopped before it could tell, and nothing on the page says whose fault) is not a failure: the job prints
it on its own line with the report's link, and the exit code does not change. The run's `summary` counts it as `look`, beside `pass`,
`fail` and `not_checked`.

Ad-hoc URLs use the team's default checks;
pass `"checks": ["add_to_cart", "search_product"]` to choose.