EngineeringAugust 1, 2026

Choosing the Right LLM for Your Task: A Decision Framework

The model selection problem

With 75+ models available through a single API, the question isn't whether you have access to good models — it's which one to pick. This guide gives you a decision framework based on real-world trade-offs.

Decision tree

1. What's your primary task?
   ├── General chat / Q&A → gpt-5.6, claude-opus-5, glm-5.2
   ├── Code generation → deepseek-v3, gpt-5.6
   ├── Long document analysis → gemini-2.5-ultra (1M ctx), claude-opus-5 (200K)
   ├── Multilingual → qwen3-72b, gemini-2.5-ultra
   ├── Creative writing → claude-opus-5, gpt-5.6
   └── Fast/simple tasks → gpt-oss-120b (Cerebras), llama-3.3-70b (Groq)

2. What's your latency budget?
   ├── < 1s (real-time) → Cerebras or Groq models
   ├── 1-5s (interactive) → Any cloud model
   └── 5-30s (batch) → Any model, prioritize cost

3. What's your cost budget?
   ├── Minimize cost → Self-hosted glm-5.2, deepseek-v3
   ├── Balanced → gpt-5.6, claude-sonnet-4-6
   └── Best quality → gpt-5.6, claude-opus-5

4. Do you need vision?
   ├── Yes → gpt-5.6, gemini-2.5-ultra, claude-opus-5
   └── No → Any model

5. What context length do you need?
   ├── < 32K → Any model
   ├── 32K-128K → gpt-5.6, glm-5.2, claude-opus-5
   ├── 128K-200K → claude-opus-5
   └── 200K-1M → gemini-2.5-ultra

Model tiers explained

Tier 1: Frontier models

These are the most capable models from leading AI labs. Use them when quality matters more than cost.

gpt-5.6 (OpenAI)

Best general-purpose model
Excellent at reasoning, coding, and creative tasks
128K context window
Strong function calling and tool use
Vision capable

claude-opus-5 (Anthropic)

Best for long-form analysis and coding
200K context window (longer than GPT)
Excellent at following complex instructions
Strong at structured output (JSON, XML)

gemini-2.5-ultra (Google)

1M token context window — longest available
Best multimodal model (images, video, audio)
Strong at multilingual tasks
Good for research and academic use

grok-4 (xAI)

Real-time knowledge (trained on X data)
Good for conversational, casual use cases
Strong at current events and trends

Tier 2: Open-weight models

These models have publicly available weights and are served by multiple providers. They offer better cost-to-quality ratios.

glm-5.2 (Z.ai / self-hosted)

Excellent general-purpose model
Available self-hosted for maximum cost savings
128K context window
Competitive with frontier models on most tasks

deepseek-v3 (DeepSeek)

Best in class for coding and math
Very cost-effective
64K context window
Strong reasoning capabilities

qwen3-72b (Alibaba)

Best for multilingual (especially Asian languages)
Cost-effective for production
32K context window

mistral-large-2 (Mistral)

Good European language support
Strong at instruction following
128K context window

Tier 3: Fast models

These models prioritize speed over absolute quality. Use them when latency is critical.

gpt-oss-120b on Cerebras (~800 tokens/second)

Fastest inference available
Good quality for most tasks
Ideal for real-time applications

llama-3.3-70b on Groq (~500 tokens/second)

Very fast inference
Good general-purpose quality
Great for chat and simple Q&A

Cost comparison

Approximate costs per million tokens (check pricing page for current rates):

| Model | Input $/1M | Output $/1M | Best for | |---|---|---|---| | gpt-5.6 | $1.40 | $4.40 | General purpose | | claude-opus-5 | $5.00 | $15.00 | Complex analysis | | gemini-2.5-ultra | $2.50 | $7.50 | Long context | | glm-5.2 | $0.50 | $1.50 | Cost-effective | | deepseek-v3 | $0.27 | $1.10 | Coding | | gpt-oss-120b (Cerebras) | $0.60 | $1.80 | Fast inference | | llama-3.3-70b (Groq) | $0.59 | $0.79 | Budget option |

Real-world scenarios

Scenario 1: Customer support chatbot

Requirements: < 2s response, cost-sensitive, 24/7 availability

Recommendation: glm-5.2 (self-hosted or cloud) or gpt-oss-120b on Cerebras

Both offer fast response times and good quality at low cost
Use a tier 1 model (gpt-5.6) for escalation to human agents

Scenario 2: Code review assistant

Requirements: High accuracy, reads large codebases, structured output

Recommendation: claude-opus-5 or deepseek-v3

Claude's 200K context window handles large files
DeepSeek is specialized for code and more cost-effective
Use gpt-5.6 as fallback

Scenario 3: Document analysis pipeline

Requirements: Process 500-page PDFs, extract structured data, batch processing

Recommendation: gemini-2.5-ultra (1M context) or claude-opus-5 (200K context)

Gemini handles the longest documents in a single pass
Claude is better at structured extraction
Batch processing reduces costs

Scenario 4: Real-time translation

Requirements: < 500ms response, multilingual, high throughput

Recommendation: gpt-oss-120b on Cerebras or qwen3-72b

Cerebras offers the fastest inference
Qwen3 has the best multilingual coverage
Use streaming for perceived performance

Scenario 5: Content moderation

Requirements: Fast, consistent, 24/7, low cost

Recommendation: gpt-oss-120b on Cerebras or llama-3.3-70b on Groq

Both are fast and cheap
Use a tier 1 model for edge cases
Consider fine-tuning for domain-specific content

Fallback strategy

Don't depend on a single model. Implement a fallback chain:

fallback_chains = {
    "gpt-5.6": ["claude-opus-5", "glm-5.2", "deepseek-v3"],
    "claude-opus-5": ["gpt-5.6", "glm-5.2"],
    "gemini-2.5-ultra": ["gpt-5.6", "claude-opus-5"],
}

When the primary model fails (timeout, rate limit, provider outage), automatically fall back to the next model in the chain.

Multi-model routing

For production systems, route requests to different models based on task type:

def select_model(task_type, complexity="medium"):
    if task_type == "code":
        return "deepseek-v3" if complexity == "low" else "gpt-5.6"
    elif task_type == "analysis":
        return "claude-opus-5" if complexity == "high" else "gpt-5.6"
    elif task_type == "chat":
        return "glm-5.2" if complexity == "low" else "gpt-5.6"
    elif task_type == "vision":
        return "gpt-5.6"
    elif task_type == "long_document":
        return "gemini-2.5-ultra"
    else:
        return "gpt-5.6"  # safe default

Testing before committing

Before choosing a model for production:

Benchmark on your data — Don't rely on published benchmarks. Test with your actual prompts and expected outputs.
Measure latency — Test under realistic load, not just single requests.
Check streaming behavior — Some models stream differently than others.
Evaluate edge cases — Test with empty inputs, very long inputs, and adversarial inputs.
Monitor cost — Track actual token usage, not just estimates.

Getting started

All 75+ models are available through a single API key at Riven. Start with gpt-5.6 as your default, then experiment with alternatives to find the best fit for your use case.