Model Selection Guide

Supported providers

Autohand connects to six model providers out of the box. Each provider serves a different use case, from cloud-hosted frontier models to fully local inference on your own hardware.

OpenRouter is the default provider because it gives you immediate access to the widest range of models through a single API key. You can change the default at any time.

Choosing a provider

The best provider depends on what matters most to your workflow. Here is a simple decision tree:

Privacy is your top concern

If your code cannot leave your machine, use a local provider. Ollama is the easiest to set up. llama.cpp gives you more control over performance tuning. MLX is purpose-built for Apple Silicon Macs and uses the GPU directly.

# Quick local setup with Ollama
brew install ollama
ollama pull llama3.3
autohand config set provider ollama
autohand config set model llama3.3

You want the best quality output

For the most capable models, use OpenRouter with Claude Opus, Claude Sonnet, or GPT-4o. These models handle complex reasoning, large codebases, and subtle refactoring tasks better than smaller alternatives.

# Use Claude Sonnet through OpenRouter
autohand config set provider openrouter
autohand config set model nvidia/nemotron-3-super-120b-a12b:free

You need the cheapest option

OpenRouter lets you pick from models at every price point. Smaller models like Llama 3.1 8B or Mistral 7B cost a fraction of what frontier models charge, and they handle routine tasks well. For zero cost, run any of these same models locally with Ollama.

Enterprise or team deployment

LLM Gateway provides a unified API that your team can share. It supports routing, rate limiting, and access control across multiple backend providers. This is useful when you need centralized billing or model governance.

# Point Autohand at your gateway
autohand config set provider llmgateway
autohand config set llmgateway.endpoint https://gateway.your-company.com/v1

Switching models

Autohand gives you three ways to change models. You can pick whichever one fits the moment.

Slash command (mid-session)

Switch models without restarting your session:

# From interactive mode
/model nvidia/nemotron-3-super-120b-a12b:free

# See all available models for your provider
/model list

CLI flag (per-session)

Set the model when you launch Autohand:

# Start a session with a specific model
autohand --model openai/gpt-4o "Review this pull request"

# Combine with pipe mode for scripts
cat error.log | autohand --model nvidia/nemotron-3-super-120b-a12b:free -y "Explain this error"

Config file (default)

Set a persistent default in your config file:

// ~/.autohand/config.json
{
  "provider": "openrouter",
  "model": "nvidia/nemotron-3-super-120b-a12b:free",
  "fallbackModel": "anthropic/claude-haiku-3"
}

The fallbackModel is used when the primary model is unavailable or when token limits are exceeded. This keeps your session running even during provider outages.

Cloud vs local tradeoffs

There is no single right answer for every situation. The table below summarizes the key differences to help you decide.

Many developers use a mixed approach: cloud models for complex architecture work and code reviews, local models for quick edits and sensitive codebases.

Provider-specific setup

Follow these step-by-step instructions to configure each provider.

OpenRouter

OpenRouter is a routing service that gives you access to models from Anthropic, OpenAI, Meta, Google, Mistral, and dozens of other providers through a single API key.

# 1. Get an API key from openrouter.ai
# 2. Set the key in your environment
export OPENROUTER_API_KEY="sk-or-..."

# Or store it in Autohand config
autohand config set provider openrouter
autohand config set openrouter.apiKey sk-or-...

# 3. Pick a model
autohand config set model nvidia/nemotron-3-super-120b-a12b:free

# 4. Verify the connection
autohand --prompt "Say hello" --max-tokens 10

OpenRouter charges per token and passes through the original provider's pricing. You can set spending limits in their dashboard.

OpenAI

Use OpenAI directly when you want GPT-4o or o1 without going through a router.

# 1. Get an API key from platform.openai.com
export OPENAI_API_KEY="sk-..."

# 2. Configure Autohand
autohand config set provider openai
autohand config set model gpt-4o

# 3. Verify
autohand --prompt "Say hello" --max-tokens 10

Ollama

Ollama makes running local models simple. It handles model downloads, quantization, and serving behind a local API.

# 1. Install Ollama
# macOS
brew install ollama

# Linux
curl -fsSL https://ollama.ai/install.sh | sh

# 2. Start the Ollama server
ollama serve

# 3. Pull a coding model
ollama pull llama3.3
ollama pull qwen2.5-coder:32b

# 4. Configure Autohand
autohand config set provider ollama
autohand config set model llama3.3
autohand config set ollama.endpoint http://localhost:11434

Ollama requires enough RAM to hold the model in memory. For a 7B parameter model, plan for 8 GB. For 32B models, you need at least 32 GB.

llama.cpp

llama.cpp is a high-performance inference server for GGUF model files. It gives you direct control over quantization levels, context sizes, and GPU layer offloading.

# 1. Install llama.cpp
brew install llama.cpp

# Or build from source
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp && make -j

# 2. Download a GGUF model file
# (from huggingface.co or similar)

# 3. Start the server
llama-server \
  --model ./models/codellama-34b-q4_k_m.gguf \
  --ctx-size 8192 \
  --n-gpu-layers 35 \
  --port 8080

# 4. Configure Autohand
autohand config set provider llamacpp
autohand config set llamacpp.endpoint http://localhost:8080

MLX (Apple Silicon)

MLX is Apple's machine learning framework. It runs models directly on the unified memory of M1, M2, M3, and M4 chips, which means no copying between CPU and GPU memory.

# 1. Install MLX and the LM server
pip install mlx-lm

# 2. Start the server with a model
mlx_lm.server --model mlx-community/Qwen2.5-Coder-32B-Instruct-4bit

# 3. Configure Autohand
autohand config set provider mlx
autohand config set mlx.endpoint http://localhost:8080

MLX works best on Macs with 32 GB or more of unified memory. With 64 GB you can run 32B parameter models at 4-bit quantization comfortably.

LLM Gateway

LLM Gateway is a self-hosted or managed proxy that routes requests to multiple backend providers. It is designed for teams that need centralized access control, usage tracking, and failover between providers.

# 1. Deploy your gateway (or use a managed instance)
# 2. Configure Autohand to use it
autohand config set provider llmgateway
autohand config set llmgateway.endpoint https://gateway.your-company.com/v1
autohand config set llmgateway.apiKey your-gateway-key

# 3. The gateway handles model routing
# Models are configured on the gateway side
autohand config set model default

Model recommendations by task

Different tasks have different requirements. Here are practical recommendations based on what we have seen work well.

You do not need to stick with one model. Use the /model command to switch mid-session when the task changes. Start with a fast model for exploration, then switch to a stronger model when you need precision.

Search engine providers

Autohand can search the web during sessions to find documentation, error explanations, and package information. Three search providers are available.

DuckDuckGo (default)

Works out of the box with no configuration. No API key required. Good for general searches and error lookups.

# Already configured by default
autohand config set search.provider duckduckgo

# Use in a session
autohand "Search for how to fix EACCES permission error in npm"

Brave Search

Offers 2,000 free API calls per month. Returns richer results than DuckDuckGo including code snippets and structured answers.

# 1. Get a free API key from brave.com/search/api
# 2. Configure
autohand config set search.provider brave
autohand config set search.brave.apiKey BSA-...

Parallel.ai

A deep research search provider that synthesizes information from multiple sources. Best for complex questions that need more than a simple search result.

# 1. Get an API key from parallel.ai
# 2. Configure
autohand config set search.provider parallel
autohand config set search.parallel.apiKey par-...

Tips

• Start with OpenRouter. It gives you access to every major model through one API key. You can always specialize later.
• Switch models mid-session. Use /model to swap from a fast model to a powerful one when you need deeper analysis without restarting.
• Use local models for sensitive code. If you are working on proprietary algorithms or customer data, Ollama and llama.cpp keep everything on your machine.
• Set a fallback model. If your primary model hits rate limits or goes down, the fallback keeps your session alive.
• Match the model to the task. You would not use a dump truck for grocery shopping. Small models are faster and cheaper for simple edits. Save the powerful models for complex work.
• Check provider status pages. If a cloud model is slow or returning errors, check the provider's status page before troubleshooting your own setup.
• Keep Ollama models updated. Run ollama pull model-name periodically to get the latest quantization improvements.