LLMGateway Integration - Autohand Docs

Overview

LLMGateway.io provides a unified interface to multiple AI providers with enterprise features. When integrated with Autohand, you get:

Access to OpenAI, Anthropic, Google, and more through one API
Intelligent request routing and load balancing
Response caching for cost optimization
Automatic failover between providers
Detailed analytics and observability
Rate limiting and budget controls

Enterprise ready: LLMGateway is designed for teams that need centralized control over AI usage, cost management, and compliance features.

Setup

Get started with LLMGateway.

Get your API key

Go to llmgateway.io and create an account
Configure your upstream providers (OpenAI, Anthropic, etc.)
Navigate to API Keys and create a key

Configure Autohand

# Set environment variable
export LLMGATEWAY_API_KEY="lgw-xxxxxxxxxxxxxxxxxxxx"

# Or configure via CLI
autohand config set llmgateway.apiKey "lgw-xxxxxxxxxxxxxxxxxxxx"

Verify your configuration:

# Start with LLMGateway provider
autohand --provider llmgateway --model gpt-4o

# Test with a prompt
autohand --prompt "Hello, which model are you?"

CLI configuration

Configure LLMGateway in your ~/.autohand/config.json:

{
  "provider": "llmgateway",
  "llmgateway": {
    "apiKey": "${LLMGATEWAY_API_KEY}",
    "baseUrl": "https://api.llmgateway.io/v1",
    "model": "gpt-4o",
    "maxTokens": 4096,
    "temperature": 0.7,
    "caching": true,
    "fallbackModels": ["z-ai/glm-4.7", "gemini-pro"]
  }
}

Configuration options

Option	Description	Default
`apiKey`	Your LLMGateway API key	-
`baseUrl`	Gateway endpoint	`https://api.llmgateway.io/v1`
`model`	Default model to use	`gpt-4o`
`caching`	Enable response caching	`true`
`fallbackModels`	Models to try if primary fails	`[]`
`maxTokens`	Maximum response tokens	`4096`

Features

Smart routing

LLMGateway can automatically route requests based on model availability, cost, or latency:

{
  "llmgateway": {
    "routing": {
      "strategy": "cost-optimized",
      "providers": ["openai", "anthropic", "google"]
    }
  }
}

Response caching

Cache identical requests to reduce costs:

{
  "llmgateway": {
    "caching": true,
    "cacheTTL": 3600
  }
}

Automatic failover

Automatically switch providers when one fails:

{
  "llmgateway": {
    "model": "gpt-4o",
    "fallbackModels": [
      "z-ai/glm-4.7",
      "gemini-1.5-pro"
    ]
  }
}

Available models

Access any model from your configured upstream providers. Common options include:

Provider	Models
OpenAI	gpt-4o, gpt-4-turbo, o1
Anthropic	z-ai/glm-4.7, claude-3-opus
Google	gemini-1.5-pro, gemini-1.5-flash
Meta	llama-3.1-70b, llama-3.1-405b

Best practices

Configure fallbacks: Set up multiple fallback models for reliability.
Enable caching: Reduce costs for repeated queries.
Set budgets: Configure spending limits in the LLMGateway dashboard.
Monitor usage: Use the analytics dashboard to track costs and usage.

Resources

Troubleshooting

Common issues

Issue	Solution
Invalid API key	Verify key at llmgateway.io
Upstream error	Check provider configuration in dashboard
Rate limited	Check rate limit settings

Debug mode

# Enable verbose logging
AUTOHAND_DEBUG=true autohand --provider llmgateway