API load testing (step-by-step)

Overview

How to run synthetic traffic against the API for metrics and SLO checks.

Why run load-style traffic

• Unit tests (make test) check contracts. A load run shows 2xx / 4xx / 5xx mix, latency, and errors in Grafana or Prometheus.
• This does not replace pytest: the runner sends real HTTP; user-create flows write to the DB.

Environment setup

1. Virtualenv: make venv && make install (or equivalent).
2. .env from env/example (make env-init), with valid APP_HOST and APP_PORT.
3. Migrations: make migrate.
4. For GET-by-id scenarios (optional scenarios/user/get.py), set ROTATE_SYSTEM_USER_IDS to real system_user_id values and adjust SHARE_OF_GROUP between user/create.py and user/get.py so they sum to 1.0 per user group.

Simulating HTTP 500 (local / non-prod only)

Normal routes rarely return a steady 500. To emit 5xx from the same API process (so http_requests_total counts them), set in .env:

LOADTEST_HTTP_500=true

After restarting the app, only the internal route GET /__loadtest/http500 is available (hidden from OpenAPI). Do not enable this flag in production.

Starting the API

From the repository root (adjust the path):

cd /path/to/study_app
        source .venv/bin/activate
        make run

If you also want Grafana and Prometheus locally, either run make observability-up in another terminal (while make run keeps the API in the first), or use make run-project to bring up the Docker stack and then start the API in the foreground (requires Docker). Details: Local development.

In a second terminal, run the traffic generator (see below).

Generator configuration (Python)

Scenarios live under tools/load_testing/scenarios/. See tools/load_testing/README.html for the full contract (with diagrams). Summary:

• scenarios/weights.py — GROUP_WEIGHTS (e.g. user, observability_5xx).
• Each scenario module exports GROUP, SHARE_OF_GROUP, MIX (sums to 1.0 within the file), and SCENARIOS (scenario key → callable building a request).
• Validation failures for user create are typically 422 — see user_payload.py and user/create.py.
• HTTP 500 without touching the DB: small share via scenarios/observability/http500.py and LOADTEST_HTTP_500=true.

Commands

Plan only (no network):

python -m tools.load_testing.runner --dry-run

Full run (default 100 requests; set LOAD_TEST_BASE_URL / LOAD_TEST_API_KEY if needed):

python -m tools.load_testing.runner --total-requests 200

Optional delay between requests:

python -m tools.load_testing.runner --total-requests 200 --delay-ms 50

Layout

tools/load_testing/
          runner.py
          README.html
          scenarios/
          weights.py
          user/create.py
          user/get.py          # optional; set ROTATE_SYSTEM_USER_IDS
          observability/http500.py
          conspectus/README.txt

Observability

Start the stack before or alongside the API (make run-project or make observability-up). After the traffic run, open Grafana (port from .env, default 3001) and the Study App Observability dashboard; in Prometheus, /alerts when SLO rules are configured.

Common issues

• Connection refused — API not running or wrong base_url.
• Script errors on “#” — a comment was passed as an argument; run the command without a trailing # ....
• Expected 200, got 404 on user GET — fix ROTATE_SYSTEM_USER_IDS in user/get.py.
• 429 Too Many Requests — lower intensity (--delay-ms) or raise the dev rate limit.

See also

• ADR 0012 — testing strategy
• ADR 0011 — SLOs and alerts
• Local development
• Developer guides index

Page history