Payload logging

Payload logging stores the full request and response bodies of a request and attaches them to its usage event by request id. Use it to see exactly what a client sent and what the provider returned. It is off by default, and nothing is captured until both an operator switch and the request’s policy allow it. Bodies are stored as Relay received and sent them, after decompression. Streamed responses are captured as the stream of frames the upstream sent. Capture happens after the response, in the background; captured bodies wait in a queue bounded by bytes (RELAY_PAYLOAD_QUEUE_BYTES, 1/16 of GOMEMLIMIT by default; see Configuration), and when it is full new captures are dropped and counted in relay_records_lost_total{kind="payload"}. Requests refused before reaching an upstream keep their captures in a small queue of their own (RELAY_PAYLOAD_REFUSED_QUEUE_BYTES, 2 MiB), written after the served ones and counted as kind="payload_refused" when dropped.

Bodies hold prompts and completions, which are often sensitive. Turn capture on for the policies you need, set a maxBytes cap, and set a retention.

Enabling capture

Turn on the payload-logging section

PUT /api/settings/payload-logging with enabled: true and a backend. While enabled is false nothing is captured, whatever the policies say.

Opt in on the policy

Set payloadLoggingEnabled: true on the policy that governs the traffic. The policy decides, whatever the relay key’s own payloadLoggingEnabled says. Only a relay key that no policy governs opts in with its own flag.

The governing policy is the one inference resolves: the relay key’s own policy, else its service account’s, else the first of the project’s policy bindings, in priority order, that names the caller. The rule is the same for normal and proxy-mode requests and for WebSocket frames. To capture everything a policy governs, set the flag on that policy. To capture a single key’s traffic, give that key a policy of its own with the flag set.

# 1. Turn capture on and pick a backend.
curl -X PUT http://localhost:8081/api/settings/payload-logging \
  -H "Authorization: Bearer $RELAY_ADMIN_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "enabled": true, "backend": "clickhouse", "maxBytes": 1048576, "retentionDays": 30,
    "file": {"path": "relay-payloads.jsonl"},
    "s3": {"endpoint": "", "bucket": "", "useSSL": true, "accessKey": {"kind": ""}, "secretKey": {"kind": ""}},
    "clickhouse": {}
  }'

# 2. Opt in on the policy that governs the traffic. PUT replaces the row,
#    so send back the policy as read with GET, with the flag set.
curl -X PUT http://localhost:8081/api/policies/by-id/<id> \
  -H "Authorization: Bearer $RELAY_ADMIN_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"metadata": {...}, "spec": {..., "payloadLoggingEnabled": true}}'

The settings write replaces the whole section: enabled, backend, maxBytes, retentionDays and every backend block are required, including the blocks not in use. To change one field, read the section with GET /api/settings/payload-logging and send its value back with the change. Changes apply without a restart. Only an admin may change this section.

Field	Default	Meaning
`enabled`	`false`	The operator switch.
`backend`	`file`	`file`, `s3` or `clickhouse`.
`maxBytes`	`4194304` (4 MiB)	Cap per stored body, measured on the decompressed body. A longer body is cut and flagged as truncated. `0` stores bodies whole.
`retentionDays`	`30`	How long the `clickhouse` backend keeps bodies; `0` keeps them forever.
`file.path`	`relay-payloads.jsonl`	File for the `file` backend.
`s3.*`		See S3.
`clickhouse.walDir`, `clickhouse.dedup`		See ClickHouse.

What is never captured

Requests whose governing policy does not have payloadLoggingEnabled, and any request while the section is off.
Anonymous proxy traffic, which has no relay key.
Batch items.
Usage events themselves: /api/usage/* and /api/logs never carry bodies, only the detail endpoint below does.

Message content reported over OpenTelemetry follows its own switch; see Message content.

Backends

File

JSONL on local disk at file.path. It is never pruned, each pod writes and reads its own file, and the read path scans the whole file. Use it for a single node or a trial only.

S3

One JSON object per request at <prefix>/<YYYY>/<MM>/<DD>/<request_id>.json in an S3-compatible bucket (AWS S3, MinIO and others). Expire objects with a bucket lifecycle rule; Relay does not delete them.

Field	Meaning
`endpoint`	`host:port`, no scheme, such as `s3.amazonaws.com`.
`bucket`	Required when the backend is `s3`.
`region`, `prefix`	Optional.
`useSSL`	Must be `true` unless `RELAY_COOKIE_SECURE=false`.
`accessKey`, `secretKey`	Secret references, or both empty to use the S3 client’s ambient credentials (an IAM role, for example).

The credentials are references limited to this section: an env reference must name a variable starting with RELAY_PAYLOAD_S3_ (for example {"kind": "env", "env": "RELAY_PAYLOAD_S3_ACCESS_KEY"}), and a stored reference must use an id starting with payload-logging:. A stored section that breaks these rules is read as the default (capture off, file backend) and logged at startup; rename the variables and write the section again.

ClickHouse

Bodies are stored in ClickHouse on the connection usage uses (RELAY_CH_DSN), buffered in a local write-ahead log first (clickhouse.walDir, default relay-payload-wal). retentionDays becomes the table’s TTL and is applied to the existing table whenever the section changes, so shortening it deletes older bodies. This is the only backend that supports erasure.

Deduplication

Agent clients resend the whole conversation on every turn, so most of each request body repeats earlier ones. clickhouse.dedup: true (off by default) stores each distinct message, tool list, and large field such as a system prompt once, and each request as the rest of its body plus references to them.

Bodies read back byte for byte. A body that is not a JSON object, including a truncated one, is stored whole, so pair dedup with maxBytes: 0.
Pieces are shared only between requests of the same project.
Each request also stores the SHA-256 of its body as received; a body that does not rebuild to that digest is refused with an error rather than returned.
Response bodies are not deduplicated.
Rows written before switching either way stay readable.

Reading captured payloads

The bodies come back on the request log’s detail endpoint, under payload:

curl -H "Authorization: Bearer $RELAY_ADMIN_TOKEN" \
  http://localhost:8081/api/logs/<request_id>

{
  "log": {"request_id": "…", "model": "gpt-5", "status": 200, "...": "..."},
  "payload": {
    "request_body": "{\"model\":\"gpt-5\",\"messages\":[…]}",
    "response_body": "{\"id\":\"chatcmpl-…\",…}",
    "request_truncated": false,
    "response_truncated": false
  }
}

payload is absent when the request was not captured or its body has expired. The same read scope applies as for usage: a caller who may not see the event gets 404, and a body is joined only when it was captured for the same project, principal and relay key as the event.

Erasing captured bodies

An admin can delete the bodies captured for a project, for a principal (a user or service account), or for one principal within one project:

curl -X POST http://localhost:8081/api/logs/erase \
  -H "Authorization: Bearer $RELAY_ADMIN_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"projectId": "<project id>", "principalId": "<principal id>"}'

At least one of projectId and principalId is required. The response, {"requests": N}, counts the requests whose bodies were deleted. The call is recorded in the audit log.

The call returns once ClickHouse has rewritten the data that held them, so they can no longer be read; ClickHouse deletes the replaced files from disk after old_parts_lifetime (8 minutes by default).
Usage events (tokens, cost, timing, ids) are kept.
Bodies a relay has captured but not yet delivered to ClickHouse are not affected; repeat the call after the traffic has stopped.
With clickhouse.dedup, a request of the same project that another relay stores while the erase runs can be left pointing at a deleted piece. Reading it fails with an error, never returns different bytes, until a later request of that project carries the piece again.
Only the clickhouse backend supports erasure, and only for the backend payload-logging currently names; file and s3 answer 501.

Get Started

Build with Relay

How It Works

Deploy and Operate

Observability

Security and Access

Reference

Enabling capture

What is never captured

Backends

File

S3

ClickHouse

Deduplication

Reading captured payloads

Erasing captured bodies

​Enabling capture

​What is never captured

​Backends

​File

​S3

​ClickHouse

​Deduplication

​Reading captured payloads

​Erasing captured bodies

Enabling capture

What is never captured

Backends

File

S3

ClickHouse

Deduplication

Reading captured payloads

Erasing captured bodies