Skip to content
Graphene

Building agents

# Building agents on Graphene

Notes for the human engineer wiring an agent to Graphene. The companion page
[For agents](/docs/for-agents) is written for the agent itself — point your
agent's retrieval at that one, and read this one yourself.

Graphene is OpenAI-compatible, so the integration is mostly boring. The
interesting parts are the ones specific to agents: cost per step compounds,
latency compounds worse, and an agent that can spend money needs a mandate, not
just a key.

## Wiring it up

Every framework below takes an OpenAI-compatible base URL. That is the whole
integration.

**LangChain / LangGraph**

```python
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    base_url="https://api.graphene.ai/v1",
    api_key=os.environ["GRAPHENE_API_KEY"],
    model="claude-opus-4-8",
)
```

**CrewAI** — set `OPENAI_API_BASE=https://api.graphene.ai/v1` and
`OPENAI_API_KEY=$GRAPHENE_API_KEY`, then use models by name as usual.

**AutoGen** — in your `config_list`, set `base_url` to
`https://api.graphene.ai/v1` and `api_key` to your Graphene key.

**Vercel AI SDK** — use the OpenAI provider with `baseURL` set to
`https://api.graphene.ai/v1`.

**MCP** — Graphene publishes provider metadata at `/.well-known/mcp.json` with
`auth.env: "GRAPHENE_API_KEY"`. Most MCP hosts will wire the credential through
from that variable without further configuration.

Switching away is the same one-line change in reverse. Portability is
deliberate: an agent platform you cannot leave is a risk you have taken on
behalf of your users.

## Keys, scopes and workspaces

- **One key per agent deployment**, not one key per fleet. Keys are the unit of
  revocation; sharing one across deployments means an incident revokes
  everything.
- **Keys inherit, they do not grant.** A key carries the routing policy, budget
  and agreement lane of its workspace. You cannot widen an agent's capabilities
  by editing its key — that is intentional, and it is what makes a compromised
  agent key a bounded problem.
- **The secret is shown once.** Store it in your secret manager at creation
  time. Rotate via the portal or `POST /api/workspace/v1/keys/{keyId}/rotate`
  rather than trying to recover it.
- **Separate workspaces for separate blast radii.** Development, evaluation and
  production agents should not share a budget or a key.

## Cost control

An agent loop turns a per-request cost into a per-task cost you did not
estimate. Two controls are worth wiring before you ship, not after:

- **Workspace budget caps.** Set a monthly cap and a warn threshold. Read the
  current position from `GET /api/budget/snapshot` — it returns
  `monthly_cap_usd`, `spend_mtd_usd` and `warn_threshold_percent`. When
  utilisation reaches 100%, inference is paused rather than silently continuing
  to bill.
- **A step budget inside the agent.** The workspace cap is a backstop measured
  in dollars per month; your agent also needs a ceiling measured in steps or
  tokens per task. Without one, a single looping task can consume the monthly
  cap legitimately.

Billing is per token at 20% under each provider's list price, with no platform
fee — so your cost model is just token count times the discounted rate. Usage
shows the routed model, token counts and the applied rate per request.

## Latency

For multi-hop agents, consistency matters more than peak speed: a planner that
sometimes waits 2 seconds is harder to design around than one that always waits
600 milliseconds. Read `/api/benchmarks` for measured time-to-first-token,
throughput and queue depth rather than trusting a marketing figure — and treat a
`null` metric as unknown, not as zero.

Practical measures:

- **Stream everything user-visible.** Time-to-first-token is what a person
  perceives; total time is what your budget pays for. They are different
  problems.
- **Keep prompt prefixes stable.** Stable prefixes across steps let caching do
  its job. Reordering a system prompt between steps discards that benefit.
- **Size the model to the step.** Most agent steps are routing, extraction or
  classification and do not need a frontier model. Reserve the expensive model
  for the step that actually needs it.

## Failure handling

Branch on the error `code`, never on `title` or `detail` — the prose can change,
the code will not.

| Code                              | Do this                                                                     |
| --------------------------------- | --------------------------------------------------------------------------- |
| `rate_limited` (429)              | Back off for `retry_after` seconds. Do not retry immediately.               |
| `unauthorized` / `invalid_token`  | Stop. Rotating a key mid-run will not fix a wrong one; fail loudly.         |
| `insufficient_scope` (403)        | Configuration error. Surface it — retrying cannot help.                     |
| `model_catalogue_access_required` | The capability needs an enterprise agreement. Escalate, do not work around. |
| `no_compliant_route` (503)        | **Terminal.** See below.                                                    |
| `budget_unavailable` (503)        | Budget service unreachable. Fail closed rather than assuming headroom.      |

**`no_compliant_route` deserves special handling.** It means no available
capacity satisfied the compliance constraints bound to your identity. Graphene
fails closed — it will not reroute a constrained workload into another
jurisdiction to make the request succeed. An immediate retry fails identically.
Alert on it; do not put it in a retry loop.

## Observability

Log per step, not per task: model, prompt and completion tokens, routed rate,
latency, and the error code if any. Two reasons. First, agent cost regressions
show up as a change in step count long before they show up as a change in the
monthly bill. Second, when something goes wrong you will want to replay the
specific step, and you cannot reconstruct it from a task-level log.

## Letting an agent pay for itself

Graphene supports x402 — an agent with a wallet can pay per request without an
API key or an account. It is a **testnet preview on Base Sepolia**: build
against it, do not put production spend on it. The six-step flow is documented
in [For agents](/docs/for-agents).

If you are designing toward that model, decide these before you enable a wallet:

- **Custody.** Where does the agent's key live, who can rotate it, and what
  happens when the process is compromised? A signing key in an agent's
  environment is a bearer instrument.
- **Mandate, not balance.** Fund the wallet to the task, and encode a spending
  ceiling in the agent as well. A balance is not authorisation.
- **Reconciliation.** Every purchase must land in a log you can audit. If your
  agent is the only record of what it bought, you do not have a record.
- **Regulatory posture.** Stablecoin settlement by autonomous counterparties
  sits inside real obligations — in Australia, AUSTRAC digital-currency-exchange
  registration and transaction monitoring. Production settlement is gated on
  that work, which is exactly why the preview is testnet-only.

## Testing

- **Test the failure paths first.** `429`, `503 no_compliant_route`, and budget
  exhaustion are the ones that will actually happen in production; the happy
  path is the easy one.
- **Pin your model in evaluation runs.** Comparing agent versions against a
  moving model is comparing nothing. The public API routes by name; if you need
  a hard guarantee that a specific model served a request, that is a
  model-pinned key and needs an enterprise agreement.
- **Assert on the manifest, not on this page.** If your integration depends on
  the model list or the pricing basis, read
  `/.well-known/agent.json` at build time. Documentation drifts; the manifest is
  generated.

## When to escalate to an agreement

Self-serve covers a lot, but some things are bound to a signed agreement rather
than a credential. If your agent needs region-constrained routing, private
routing without shared-tenant capacity, model pinning, bespoke retention,
attestation and replay, or any latency or uptime commitment, that is an
enterprise or sovereign agreement. See
[Enterprise and sovereign access](/docs/enterprise-access).

Designing around the gap is the wrong move — the constraints exist because
someone's compliance obligation depends on them holding.

## Next steps

- **[API reference](/docs/api-reference)** — endpoints, errors, rate limits
- **[Agent cards](/docs/agent-cards)** — the machine-readable descriptors
- **[For agents](/docs/for-agents)** — the page to give your agent
- **[Routing and budgets](/docs/routing-and-budgets)** — workspace policy
  controls