Go SDK

The Go SDK (github.com/wyolet/relay/sdk) takes one request in Relay’s canonical shape and sends it through Relay or straight to a provider. Its module imports nothing from the server, so it can be vendored on its own.

go get github.com/wyolet/relay/sdk

Calling Relay

client.Relay sends requests to Relay’s POST /v1/generate with your relay key. Routing, policy, rate limits and key pooling happen on the server.

import (
	"github.com/wyolet/relay/sdk/client"
	v1 "github.com/wyolet/relay/sdk/v1"
)

c := client.Relay("https://relay.example.com", os.Getenv("WR_API_KEY"))

resp, err := c.Generate(ctx, &v1.Request{
	Model: v1.ModelRefs{"claude-sonnet-4-6"},
	Input: []v1.Item{&v1.Message{Role: v1.RoleUser, Content: []v1.Part{&v1.TextPart{Text: "Hello"}}}},
})
if err != nil {
	log.Fatal(err)
}
for _, item := range resp.Output {
	if m, ok := item.(*v1.Message); ok {
		for _, p := range m.Content {
			if t, ok := p.(*v1.OutputTextPart); ok {
				fmt.Print(t.Text)
			}
		}
	}
}
fmt.Println(resp.Usage["input"], resp.Usage["output"])

GenerateStream returns the canonical event stream. Call Recv until io.EOF and close the stream when done:

s, err := c.GenerateStream(ctx, req)
if err != nil {
	log.Fatal(err)
}
defer s.Close()
for {
	ev, err := s.Recv()
	if err == io.EOF {
		break
	}
	if err != nil {
		log.Fatal(err)
	}
	if ev.Type == v1.EventItemDelta {
		var d v1.ItemDeltaEvent
		_ = json.Unmarshal(ev.Data, &d)
		fmt.Print(d.Delta)
	}
}

A non-2xx answer is returned as *client.APIError, with StatusCode, Code, Message and the raw body. Missing or invalid configuration does not fail construction: it is returned by the first Generate or GenerateStream.

Relay client environment

Empty arguments to client.Relay fall back to the environment:

Variable	Meaning
`WR_BASE_URL`	Relay’s inference URL, used when `baseURL` is empty
`WR_API_KEY`	Relay key, used when `relayKey` is empty
`WR_USAGE`	Default `X-WR-Usage` header. `full` adds the `relay_usage` block (request id, tokens, attempts, timing) to every response.
`WR_HEADERS`	Extra headers on every request, as `name1=value1,name2=value2`. Options passed to `client.Relay` override them.
`WR_TIMEOUT`	Deadline for `Generate`, as a Go duration (`30s`). Streams are not affected.

client.WithHeader(name, value) adds a header per client, for example X-WR-Request-Tags to tag usage events. See Headers.

WebSocket

client.RelayWS has the same API as client.Relay but sends every request over one WebSocket to Relay’s /v1/ws, so a loop of many requests pays the TLS and authentication handshake once. One client sends one request at a time: a request holds the connection until its response or stream is closed. Call c.Close() when done. See WebSocket for the protocol.

c := client.RelayWS("https://relay.example.com", os.Getenv("WR_API_KEY"))
defer c.Close()

Calling a provider directly

client.For resolves a model reference against the catalog embedded in the SDK and returns a client wired to that host’s API, shape and pricing. It calls the provider with your provider key, without Relay:

c, err := client.For("claude-sonnet-4-6", os.Getenv("ANTHROPIC_API_KEY"))
resp, err := c.Generate(ctx, req)
cost, ok := resp.Cost() // in the binding's price currency; ok is false when the binding is unpriced

References take the forms gpt-4o, openai/gpt-4o and gpt-4o@<host>. Cost() is available on responses and, once the final usage event has arrived, on streams; it is always false for clients that call Relay, since Relay prices the call itself. client.OpenAI, client.OpenAIResponses, client.Anthropic and client.Gemini build a client for a base URL you give, with no catalog lookup and no pricing. An empty key falls back to OPENAI_API_KEY, ANTHROPIC_API_KEY, or GEMINI_API_KEY then GOOGLE_API_KEY.

Choosing where calls go

client.ForMode builds one of three clients from a single switch, so the same code can run through Relay in one deployment and straight to the provider in another:

Mode	The call goes to	Relay records it	Message content
`proxy`	Relay’s `/v1/generate`, with your relay key	yes, as a routed request	kept when Relay’s payload logging applies
`telemetry`	the provider, with your provider key	yes, reported over OTLP with your relay key	sent when Relay asks for it, or when you opt in and no policy decides; see Content
`direct`	the provider, with your provider key	no	never sent

The SDK’s proxy mode is a normal routed request through Relay, not Relay’s proxy mode, which forwards your own upstream key. Telemetry cannot be on in proxy mode, so a call is never counted twice.

c, err := client.ForMode("gpt-4o")
if err != nil {
	log.Fatal(err)
}
defer c.Close() // in telemetry mode, sends the calls still queued

resp, err := c.Generate(ctx, &v1.Request{
	Input: []v1.Item{&v1.Message{Role: v1.RoleUser, Content: []v1.Part{&v1.TextPart{Text: "Hello"}}}},
})

How the mode is chosen

The mode is, in order:

client.WithMode(client.ModeTelemetry) (or ModeProxy, ModeDirect).
WR_MODE = proxy, telemetry or direct.
proxy when WR_BASE_URL and WR_API_KEY are set, else direct. Telemetry is never chosen by default.

In proxy mode the client is client.Relay with WR_BASE_URL and WR_API_KEY, and the model reference passed to ForMode is the model Relay resolves. In telemetry and direct mode the reference is resolved against the catalog embedded in the SDK, like client.For, and the call goes to that host.

Environment

Variable	Used by	Meaning
`WR_MODE`	`ForMode`	`proxy`, `telemetry` or `direct`
`WR_BASE_URL`, `WR_API_KEY`	proxy, telemetry	Relay’s inference URL and a relay key. Telemetry exports to `WR_BASE_URL` + `/otlp` with this key.
`OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, `GEMINI_API_KEY` (then `GOOGLE_API_KEY`)	telemetry, direct	The provider key, chosen by the resolved binding’s adapter. `client.WithProviderKey` overrides it; use it for OpenAI-compatible hosts other than OpenAI.
`OTEL_EXPORTER_OTLP_ENDPOINT`, `OTEL_EXPORTER_OTLP_HEADERS`	telemetry	Export to another OTLP/HTTP backend instead. When the endpoint comes from here, `WR_API_KEY` is not sent; put credentials in the headers variable.
`OTEL_SERVICE_NAME`	telemetry	The `service.name` of the reports.
`OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT`	telemetry	`true` (or `span_only`, `event_only`, `span_and_event`) opts in to sending message content.
`WR_CATALOG_URL`	telemetry, direct	Base URL of the catalog server that `Models` and `Catalog` read; default `https://catalog.wyolet.com`. `off` uses the catalog embedded in the SDK and makes no request.
`WR_CATALOG_CACHE_DIR`	telemetry, direct	Directory for the catalog disk cache. Without it nothing is written to disk.

Missing configuration does not panic: it is returned by the first Generate or GenerateStream.

Listing models

c.Models(ctx) returns model ids and c.Catalog(ctx) the catalog behind a model picker, with hosts, models, providers and prices. Where they read depends on the mode:

Mode	Reads	Contains
`proxy` (and `client.Relay`, `client.RelayWS`)	Relay’s `GET /v1/models` and `GET /v1/catalog`, with your relay key	what the key’s policy can route to
`telemetry`, `direct`	the latest public catalog, through `catalogsource.Latest`	the host the model reference resolved to

In telemetry and direct mode nothing is sent to Relay or to the provider; the relay key authenticates reports only. Models lists the model slugs the host serves. What the catalog lists is what the host serves, not what your provider key can call; a model the key can’t use fails on the call. A client built with client.OpenAI, client.Anthropic, client.Gemini or client.New has no catalog host and gets the whole catalog. Each result carries:

Origin: relay, catalog (the catalog server’s latest release) or embedded (the catalog compiled into the SDK).
Stale: an earlier copy, served because a refresh failed.
Err: the error behind Stale or an embedded fallback; nil when the result is current.

The returned error is non-nil only when there is nothing to show, such as a relay that refuses the key (401, 403). catalogsource.Latest reads the release index at most once every 10 minutes and downloads a release only when the index names a new one, checking its sha256. When the server can’t be read it serves, in order, the copy held in memory, the last release in WR_CATALOG_CACHE_DIR, and the embedded catalog. A release older than the embedded catalog is never used. By default releases are read from https://catalog.wyolet.com; set WR_CATALOG_URL to use another catalog server, or share one source between clients:

src := catalogsource.Latest(
	catalogsource.WithBaseURL("https://catalog.example.com"),
	catalogsource.WithCacheDir(cacheDir),
)
c, err := client.ForMode("gpt-4o", client.WithClientOptions(client.WithCatalogSource(src)))

client.For and ForMode resolve the model reference against the embedded catalog and make no request. To resolve against the latest release, pass it to client.ForFrom:

ic, err := catalogsource.Latest().Catalog(ctx)
c, err := client.ForFrom(ic, "gpt-4o", os.Getenv("OPENAI_API_KEY"))

Telemetry mode

Each call becomes one OpenTelemetry span in the GenAI semantic conventions, exported as OTLP/HTTP JSON with gzip. Token counts come from the SDK’s own translators, the same counts Relay stores for a call routed through it, including cache reads and writes, reasoning and audio, and streams. A call made through client.For or ForMode also carries wyolet.relay.host, the catalog host it was resolved to, so Relay prices it at that host. Reporting never slows or fails a call. Calls wait in a queue of 2,048 and are exported in batches of up to 512, every 5 seconds. When the queue is full a call is dropped; an export that fails is retried up to three times (on 429, 502, 503, 504 and network errors, honouring Retry-After, 30 seconds at most) and then dropped. Dropped() counts what was lost and LastWarning() returns the last message Relay sent back in a partial success. A stream is reported when it ends: at io.EOF, on a read error, or when it is closed. A stream closed before its last event is reported with the usage seen so far, error.type client_canceled and finish reason error. To share one emitter between clients, or to send reports elsewhere, build it yourself:

e := telemetry.New(
	telemetry.WithEndpoint("https://relay.example.com/otlp"),
	telemetry.WithAPIKey(os.Getenv("WR_API_KEY")),
	telemetry.WithServiceName("indexer"),
)
defer e.Close(context.Background())

c, err := client.ForMode("gpt-4o", client.WithMode(client.ModeTelemetry), client.WithEmitter(e))
// or, without ForMode: client.For("gpt-4o", key, client.WithClient(client.WithTelemetry(e)))

WithTelemetry on a client built by client.Relay or client.RelayWS is a configuration error returned by the first call.

Flush and Close

A process that exits right after its calls must send what is queued first:

e.Flush(ctx) exports every call recorded so far and returns the first export error.
e.Close(ctx) does the same and stops the emitter; calls recorded afterwards are dropped. When ctx ends first, the export in flight is abandoned, the calls still queued are dropped and counted, and the emitter stops all the same.
c.Close() closes the emitter ForMode built for the client. An emitter you passed with WithEmitter is yours to close. c.Telemetry() returns the client’s emitter either way.

Content

The policy that governs your relay key decides whether Relay stores message content; your opt-in decides only when no policy governs the key, and nothing is stored while Relay’s content capture is off. Relay answers every accepted export with the X-WR-Content-Capture header, and the SDK follows it:

Answer	Content is sent
`store`	yes, also without your opt-in
`drop`	no, also with your opt-in
`client`, or no header (another OTLP backend)	when you opted in

The emitter asks as soon as it is built, with an empty logs export, and every export refreshes the answer. An answer older than five minutes is asked again, and the last one keeps deciding until the new one arrives; a failed request keeps it too. Before the first answer your opt-in decides, so with a store answer and no opt-in, a call made within that first round trip carries no content. Content is copied from the request only when it may be sent; under drop it is never built. Content travels on the logs signal as one gen_ai.client.inference.operation.details event per call, with the same trace and span id as the call’s span: gen_ai.system_instructions, gen_ai.input.messages, gen_ai.tool.definitions and gen_ai.output.messages, in the conventions’ message schema. Opaque provider data (reasoning signatures, encrypted reasoning), citations, image detail hints, file names and MCP server headers are not sent.

Traces and conversations

ctx = telemetry.ContextWithTraceparent(ctx, r.Header.Get("traceparent")) // the call joins this trace
ctx = telemetry.ContextWithConversation(ctx, sessionID)                // gen_ai.conversation.id

Without a traceparent, each call starts a trace of its own.

Get Started

Build with Relay

How It Works

Deploy and Operate

Observability

Security and Access

Reference

Calling Relay

Relay client environment

WebSocket

Calling a provider directly

Choosing where calls go

How the mode is chosen

Environment

Listing models

Telemetry mode

Flush and Close

Content

Traces and conversations

​Calling Relay

​Relay client environment

​WebSocket

​Calling a provider directly

​Choosing where calls go

​How the mode is chosen

​Environment

​Listing models

​Telemetry mode

​Flush and Close

​Content

​Traces and conversations

Calling Relay

Relay client environment

WebSocket

Calling a provider directly

Choosing where calls go

How the mode is chosen

Environment

Listing models

Telemetry mode

Flush and Close

Content

Traces and conversations