---
description: Choose, tune, and connect the LLM that powers your agent with the useModel hook.
title: Models | Flue
image: https://flueframework.com/docs/og4.jpg
---

# Models

Last updated Jul 17, 2026[View as Markdown](https://nightly.flueframework.com/docs/guide/models/index.md)

Every agent is powered by exactly one LLM at a time, declared with `useModel()` — the single required hook in Flue. This guide covers choosing a model, tuning how it’s called, when a change takes effect, supplying provider credentials, and connecting providers Flue doesn’t know out of the box.

## Declaring a model

Call `useModel()` in your agent function’s body with a model specifier string:

```ts
'use agent';
import { useModel } from '@flue/runtime';

export function TriageAgent() {
  useModel('anthropic/claude-sonnet-4-6');
  return 'Investigate the reported issue and recommend the next action.';
}
```

`useModel()` is a declaration, not a client: it returns nothing, and you never construct an SDK object or pass an API key through your agent code. You name the model; the Flue runtime owns the connection, authentication, streaming, and retries behind it.

Two rules:

* **The call is required.** An agent render without a `useModel()` call cannot start.
* **Call it exactly once per render.** An agent has one model. The _argument_ may change from render to render (more on that [below](#changing-models-mid-conversation)), but the call itself may not disappear or repeat.

The call can live in the agent body or in a [custom hook](https://nightly.flueframework.com/docs/guide/agent-hooks/#custom-hooks) the body calls. The one place it’s not available is a subagent render — a delegate’s model is set on its `useSubagent()` definition instead, and it inherits the parent’s model when unset. See [Subagents](https://nightly.flueframework.com/docs/guide/subagents/).

## Model specifier

A model specifier is a plain string in `'provider-id/model-id'` format. Everything up to the first `/` names the provider; the rest is the provider’s own model ID, which may itself contain slashes:

* `anthropic/claude-sonnet-4-6` — provider `anthropic`, model `claude-sonnet-4-6`
* `openai/gpt-5.5` — provider `openai`, model `gpt-5.5`
* `openrouter/moonshotai/kimi-k2.6` — provider `openrouter`, model `moonshotai/kimi-k2.6`
* `cloudflare/@cf/moonshotai/kimi-k2.6` — provider `cloudflare`, model `@cf/moonshotai/kimi-k2.6`

Flue resolves specifiers against the model catalog from [Pi](https://pi.dev/docs/latest/providers), which ships entries for the major providers — `anthropic`, `openai`, `google`, `amazon-bedrock`, `google-vertex`, `groq`, `mistral`, `xai`, `deepseek`, `cerebras`, `together`, `fireworks`, `openrouter`, and more. A catalog entry carries the model’s wire protocol, endpoint, context-window size, output-token limit, cost rates, reasoning support, and accepted input modalities. That metadata decides when [compaction](#compaction) triggers, whether a [thinking level](#model-reasoning-effort) reaches the wire, and whether the model can accept images.

An unknown specifier fails fast: the run errors with the unresolved provider and model ID before any request is sent. To teach Flue a specifier the catalog doesn’t know — a local model, a proxy, a brand-new release — register it yourself; see [Custom providers](#custom-providers).

On the Cloudflare target there is one more built-in provider ID: `cloudflare/...` model specifiers run on [Workers AI](#cloudflare-workers-ai-cloudflare-only) with no API key at all.

## Model reasoning effort

`useModel()` accepts an options object as its second argument with two fields: `thinkingLevel` and `compaction`.

```ts
useModel('anthropic/claude-opus-4-6', {
  thinkingLevel: 'high',
  compaction: { keepRecentTokens: 16000 },
});
```

`thinkingLevel` sets the default reasoning effort for the agent’s model calls: `'off'`, `'minimal'`, `'low'`, `'medium'`, `'high'`, or `'xhigh'`. When you don’t set it, the runtime uses `'medium'`.

Higher levels increase reasoning depth at the cost of latency and tokens; `'off'` disables extended thinking entirely. The value is a _default_: individual operations may override it — a [subagent definition](https://nightly.flueframework.com/docs/guide/subagents/) can pin its own `thinkingLevel`, and programmatic `harness.prompt(...)` calls accept one per operation.

Thinking only reaches the wire for models marked reasoning-capable. Catalog models carry that flag already, but if you register a custom provider and skip its `reasoning` metadata, a forwarded `thinkingLevel` is silently dropped. See [Custom providers](#custom-providers).

## Compaction

Agent conversations can outlive any context window. As a conversation approaches the model’s limit, Flue automatically **compacts** it: older history is folded into a summary while recent messages stay verbatim, and the conversation continues. The `compaction` option tunes that behavior:

```ts
useModel('anthropic/claude-opus-4-6', {
  compaction: {
    // Trigger earlier or later: compaction runs when used tokens
    // exceed contextWindow - reserveTokens. Default: model-aware, ≤ 20000.
    reserveTokens: 30000,
    // How much recent history survives verbatim. Default: 8000.
    keepRecentTokens: 16000,
    // Summarize with a cheaper model than the session runs on.
    model: 'anthropic/claude-haiku-4-5',
  },
});
```

All three fields are optional; each overrides a model-aware default. Setting `compaction.model` offloads summarization to a cheaper model.

Passing `compaction: false` disables _threshold_ compaction — the automatic trigger. Overflow recovery and explicit `harness.compact()` calls still compact when the conversation no longer fits. The full field reference lives at [CompactionConfig](https://nightly.flueframework.com/docs/reference/agent-hooks-api/#compactionconfig).

## Changing models mid-conversation

The agent function re-renders before every model call, and `useModel()` runs again each time — so the specifier can be computed, not constant. A common pattern is escalation, where durable state moves an agent from a cheap model to a strong one:

```ts
'use agent';
import { useModel, usePersistentState, useTool } from '@flue/runtime';

export function Reviewer() {
  const [escalated, setEscalated] = usePersistentState('escalated', false);
  useModel(escalated ? 'anthropic/claude-opus-4-6' : 'anthropic/claude-haiku-4-5');

  useTool({
    name: 'escalate_review',
    description: 'Escalate when the change is too complex for a quick pass.',
    async run() {
      setEscalated(true);
      return 'Escalated. A stronger model will take over.';
    },
  });

  return 'Review the proposed change and leave actionable feedback.';
}
```

The model, thinking level, and compaction settings are **submission-scoped**: the runtime reads them once, when the agent wakes to process an accepted input (a _submission_ — see [Durability](https://nightly.flueframework.com/docs/guide/durability/)). A different value computed by a re-render mid-run latches and takes effect on the _next_ submission, not in the middle of the current one. In the example above, the response where `escalate_review` fires finishes on the cheap model; the conversation’s next message runs on the strong one.

There is no per-message model parameter on `dispatch(...)` or the HTTP surface: the model choice belongs to the agent function.

## Provider credentials

Model providers authenticate with API keys, and the runtime resolves them from the environment — your agent code never handles them.

### Local development

Put keys in a `.env` file at the project root, using the variable name each provider expects:

```bash
ANTHROPIC_API_KEY="sk-ant-..."
OPENAI_API_KEY="sk-..."
GEMINI_API_KEY="..."
```

`anthropic` reads `ANTHROPIC_API_KEY`, `openai` reads `OPENAI_API_KEY`, `google` reads `GEMINI_API_KEY`, `groq` reads `GROQ_API_KEY` — the pattern holds across providers (see [Pi’s provider documentation](https://pi.dev/docs/latest/providers) for the full list). Both local entry points load the file for you, with shell-exported values always winning over file values:

* [flue run](https://nightly.flueframework.com/docs/cli/run/) loads the project-root `.env`; pass `--env <path>` to select one alternate file.
* `vite dev` loads Vite’s standard file set: `.env`, `.env.local`, `.env.<mode>`, `.env.<mode>.local`.

Don’t commit `.env` files.

### Deployed environments

Deployed servers read only the real environment — no `.env` loading:

* **Node.js** — supply keys as process environment variables through your host’s secret mechanism. See [Node.js target — Environment and secrets](https://nightly.flueframework.com/docs/guide/node-target/#environment-and-secrets).
* **Cloudflare** — add each key as a Worker secret (`npx wrangler secret put ANTHROPIC_API_KEY`); locally, `.dev.vars` plays the `.env` role. See the [Cloudflare deploy guide](https://nightly.flueframework.com/docs/ecosystem/deploy/cloudflare/).

When the environment-variable convention doesn’t fit — a gateway with its own credential, a secret manager that hands you the value in code — pass `apiKey` to `registerProvider()`, which takes precedence over the environment lookup for that provider ID:

```ts
import { registerProvider } from '@flue/runtime';

registerProvider('anthropic', { apiKey: await secrets.get('anthropic-key') });
```

## Custom providers

`registerProvider()` teaches the runtime a provider ID. Call it at module top level in `app.ts`, before any agent runs. Registrations are keyed by provider ID, and each call replaces that ID’s previous registration. One placement caveat: [flue run](https://nightly.flueframework.com/docs/cli/run/) loads only the agent module, never `app.ts` — when an agent must also work under `flue run`, put the registration in the agent module instead.

For a provider ID the catalog already knows, registration is an _overlay_: models keep their catalog metadata (cost, context window, wire protocol), and your options are layered on top. That makes routing a built-in provider through a gateway or proxy one call, without changing the specifiers your agents use:

```ts
import { registerProvider } from '@flue/runtime';

registerProvider('anthropic', {
  baseUrl: 'https://gateway.example.com/anthropic',
  apiKey: process.env.GATEWAY_KEY,
});
```

For a provider ID the catalog doesn’t know, you register it from scratch. `api` (the wire protocol) and `baseUrl` are required, and any OpenAI- or Anthropic-compatible endpoint works — here’s a local Ollama server:

```ts
import { registerProvider } from '@flue/runtime';

registerProvider('ollama', {
  api: 'openai-completions',
  baseUrl: 'http://localhost:11434/v1',
  contextWindow: 128000,
  reasoning: false,
});
```

```ts
useModel('ollama/llama3.1:8b');
```

The runtime trusts the declared metadata. A model ID no catalog knows defaults to `reasoning: false` (a `thinkingLevel` is silently dropped), `input: ['text']` (attached images are replaced with an “(image omitted)” placeholder), and `contextWindow: 0` (unknown, so threshold compaction can’t engage). Set `reasoning`, `input`, `contextWindow`, and `maxTokens` at the provider level, or per model ID via the `models` map.

For a wire protocol Pi doesn’t ship at all, `registerApiProvider()` registers a custom handler that `registerProvider()` can then reference by its `api` slug. The full option tables for both functions live in the [Provider API reference](https://nightly.flueframework.com/docs/reference/provider-api/).

## Cloudflare Workers AI (Cloudflare only)

On the Cloudflare target, the `cloudflare` provider ID is registered automatically and runs models on [Workers AI](https://developers.cloudflare.com/workers-ai/) — no API key, no external account; authorization and billing follow the Worker, including the [Workers AI pricing and daily free allocation](https://developers.cloudflare.com/workers-ai/platform/pricing/):

```ts
export function Assistant() {
  useModel('cloudflare/@cf/moonshotai/kimi-k2.6');
  return 'Help the user with their question.';
}
```

Declare the `AI` binding the provider uses in your project’s Wrangler configuration:

```jsonc
{
  "$schema": "./node_modules/wrangler/config-schema.json",
  "ai": {
    "binding": "AI",
  },
}
```

Everything after `cloudflare/` is passed as the model ID to `env.AI.run(...)`. That can be a Workers AI model ID such as `@cf/moonshotai/kimi-k2.6`, or a binding-supported AI Gateway model ID such as `openai/gpt-5.5` when the Worker should reach that model through Cloudflare’s gateway path — `cloudflare/openai/gpt-5.5` bills through Cloudflare, while plain `openai/gpt-5.5` uses Flue’s direct OpenAI provider and its API key.

By default, every `cloudflare/...` call routes through Cloudflare’s [AI Gateway](https://developers.cloudflare.com/ai-gateway/), giving you caching, logging, and budget controls in the dashboard out of the box. To target a named gateway, tune caching and logging, or opt out, register the `cloudflare` provider yourself in `app.ts` — your registration wins over the generated default:

```ts
import { registerProvider } from '@flue/runtime';
import { env } from 'cloudflare:workers';

registerProvider('cloudflare', {
  api: 'cloudflare-ai-binding',
  binding: env.AI,
  gateway: { id: 'my-gateway', cacheTtl: 300, metadata: { tenant: 'acme' } },
  // ...or `gateway: false` to bypass AI Gateway entirely.
});
```

Cloudflare’s model surface is also reachable from any target through two ordinary catalog providers: `cloudflare-workers-ai/...` (URL-backed Workers AI) and `cloudflare-ai-gateway/...` (URL-backed AI Gateway), both authenticating with `CLOUDFLARE_API_KEY` like any other hosted provider. See the [Cloudflare target guide](https://nightly.flueframework.com/docs/guide/cloudflare-target/#workers-ai-and-ai-gateway) for the target’s full model story.

## Next steps

* [Agent Hooks API](https://nightly.flueframework.com/docs/reference/agent-hooks-api/#usemodel) — the full `useModel()` contract, `ThinkingLevel`, and `CompactionConfig`.
* [Provider API](https://nightly.flueframework.com/docs/reference/provider-api/) — every `registerProvider()` and `registerApiProvider()` option.
* [Subagents](https://nightly.flueframework.com/docs/guide/subagents/) — give a delegate its own model and thinking level.
* [Durability](https://nightly.flueframework.com/docs/guide/durability/) — what a submission is and how interrupted model work recovers.
* [Observability](https://nightly.flueframework.com/docs/guide/observability/) — inspect model calls, token usage, and provider diagnostics.

## Docs Navigation

Current page: [Models](https://nightly.flueframework.com/docs/guide/models/)

### Sections

* [Guide](https://nightly.flueframework.com/docs/guide/getting-started/)
* [Reference](https://nightly.flueframework.com/docs/reference/agent-api/)
* [CLI](https://nightly.flueframework.com/docs/cli/overview/)
* [SDK](https://nightly.flueframework.com/docs/sdk/overview/)
* [Ecosystem](https://nightly.flueframework.com/docs/ecosystem/)

### Introduction

* [Getting Started](https://nightly.flueframework.com/docs/guide/getting-started/)
* [Why Flue?](https://nightly.flueframework.com/docs/guide/why-flue/)
* [Migration Guide](https://nightly.flueframework.com/docs/guide/migration/)
* [Changelog](https://github.com/withastro/flue/blob/main/CHANGELOG.md)

### Guides

* [Project Layout](https://nightly.flueframework.com/docs/guide/project-layout/)
* [Agents](https://nightly.flueframework.com/docs/guide/building-agents/)
* [Agent Hooks](https://nightly.flueframework.com/docs/guide/agent-hooks/)
* [Models](https://nightly.flueframework.com/docs/guide/models/)
* [Tools](https://nightly.flueframework.com/docs/guide/tools/)
* [Skills](https://nightly.flueframework.com/docs/guide/skills/)
* [Subagents](https://nightly.flueframework.com/docs/guide/subagents/)
* [Sandboxes](https://nightly.flueframework.com/docs/guide/sandboxes/)
* [Routing](https://nightly.flueframework.com/docs/guide/routing/)
* [Database](https://nightly.flueframework.com/docs/guide/database/)

### Advanced

* [Deploy](https://nightly.flueframework.com/docs/guide/deploy/)
* [Schedules](https://nightly.flueframework.com/docs/guide/schedules/)
* [Channels](https://nightly.flueframework.com/docs/guide/channels/)
* [Evals](https://nightly.flueframework.com/docs/guide/evals/)
* [Observability](https://nightly.flueframework.com/docs/guide/observability/)
* [Durability](https://nightly.flueframework.com/docs/guide/durability/)

### Frontend

* [React](https://nightly.flueframework.com/docs/guide/react/)

### Targets

* [Cloudflare](https://nightly.flueframework.com/docs/guide/cloudflare-target/)
* [Node.js](https://nightly.flueframework.com/docs/guide/node-target/)