Models

Model management overview

Amplify combines static models (hardcoded per provider) with dynamic models (fetched at runtime from provider APIs) into a unified model list. Dynamic models override static models when they share the same name-provider key.

Default model: claude-3-5-sonnet-latest (defined in app/utils/constants.ts)

ModelInfo interface:

interface ModelInfo {
  name: string;              // Model ID (e.g., "gpt-4o", "claude-3-5-sonnet")
  label: string;             // Human-readable display name
  provider: string;          // Provider name (e.g., "OpenAI", "Anthropic")
  maxTokenAllowed: number;   // Maximum context window size
  maxCompletionTokens?: number; // Maximum output tokens (optional)
  tpm?: number;              // Tokens per minute limit
  rpm?: number;              // Requests per minute limit
}

Provider categories

Models come from three provider categories:

Cloud providers (API key required):

• OpenAI, Anthropic, Google, Deepseek, Groq, Mistral, Cohere, Together, xAI, Perplexity, Moonshot, Z.ai, HuggingFace, Hyperbolic, Fireworks, Cerebras, Github, OpenRouter, AmazonBedrock

Local providers (baseUrl required, no API key):

• Ollama : Self-hosted models, base URL http://127.0.0.1:11434
• LMStudio : Local model inference, base URL http://127.0.0.1:1234
• OpenAILike : Any OpenAI-compatible endpoint

Routing/Meta providers:

• OpenRouter : Aggregates multiple providers, shows pricing in labels (in:$X.XX out:$X.XX)

Per-model configuration

Each model can have individual configuration persisted in localStorage key amplify:model-config.

interface ModelConfig {
  thinkingEnabled: boolean;   // Master toggle for extended thinking
  budgetTokens: number;       // Token budget for thinking (default 4096)
  effort: ReasoningEffort;    // 'low' | 'medium' | 'high'
  maxOutputTokens: number;    // Hard cap on output tokens; 0 = use model default
}

interface ModelConfigStore extends ModelConfig {
  perModel: Record<string, ModelConfig>;  // Key = "Provider:ModelName"
}

Defaults: thinkingEnabled=true, budgetTokens=4096, effort='medium', maxOutputTokens=0

When you switch models, Amplify restores the per-model config from perModel["Provider:ModelName"]. This means your thinking/reasoning settings survive model switches.

Thinking and reasoning

Amplify supports extended thinking/reasoning for models that provide it. The thinking.ts module translates unified config into per-provider providerOptions.

Rate limits

Amplify implements per-provider rate limiting to prevent API quota exhaustion.

interface RateLimitConfig {
  rpm: number;              // Requests per minute (0 = unlimited)
  tpm: number;              // Tokens per minute (0 = unlimited)
  rpd: number;              // Requests per day (0 = unlimited)
  autoShrinkToTpm: boolean; // Auto-shrink context to fit TPM instead of erroring
}

Suggested defaults (from 2025-2026 official provider docs):

Rate limits are persisted in localStorage key amplify:rate-limits. The preFlightCheck() function estimates request tokens and can throttle, shrink context, or reject requests that exceed limits.

• autoShrinkToTpm

  When autoShrinkToTpm is enabled, Amplify automatically reduces the context window (by summarizing older messages) to fit within the TPM limit instead of returning an error. This is useful for providers with strict TPM limits like xAI and Groq.

Notable static models

Key models available without dynamic fetching: