---
title: "LLMGateway Integration"
source: https://docs.autohand.ai/integrations/llmgateway
---

# LLMGateway

Enterprise-grade AI gateway with unified API access to multiple providers. Smart routing, caching, and observability for production LLM applications.

## Overview

LLMGateway.io provides a unified interface to multiple AI providers with enterprise features. When integrated with Autohand, you get:

-   Access to OpenAI, Anthropic, Google, and more through one API
-   Intelligent request routing and load balancing
-   Response caching for cost optimization
-   Automatic failover between providers
-   Detailed analytics and observability
-   Rate limiting and budget controls

**Enterprise ready:** LLMGateway is designed for teams that need centralized control over AI usage, cost management, and compliance features.

## Setup

Get started with LLMGateway.

### Get your API key

1.  Go to [llmgateway.io](https://llmgateway.io) and create an account
2.  Configure your upstream providers (OpenAI, Anthropic, etc.)
3.  Navigate to **API Keys** and create a key

### Configure Autohand

``` bash
# Set environment variable
export LLMGATEWAY_API_KEY="lgw-xxxxxxxxxxxxxxxxxxxx"

# Or configure via CLI
autohand config set llmgateway.apiKey "lgw-xxxxxxxxxxxxxxxxxxxx"
```

Verify your configuration:

``` bash
# Start with LLMGateway provider
autohand --provider llmgateway --model gpt-4o

# Test with a prompt
autohand --prompt "Hello, which model are you?"
```

## CLI configuration

Configure LLMGateway in your `~/.autohand/config.json`:

``` json
{
  "provider": "llmgateway",
  "llmgateway": {
    "apiKey": "${LLMGATEWAY_API_KEY}",
    "baseUrl": "https://api.llmgateway.io/v1",
    "model": "gpt-4o",
    "maxTokens": 4096,
    "temperature": 0.7,
    "caching": true,
    "fallbackModels": ["z-ai/glm-4.7", "gemini-pro"]
  }
}
```

### Configuration options

| Option | Description | Default |
|---|---|---|
| apiKey | Your LLMGateway API key | - |
| baseUrl | Gateway endpoint | https://api.llmgateway.io/v1 |
| model | Default model to use | gpt-4o |
| caching | Enable response caching | true |
| fallbackModels | Models to try if primary fails | [] |
| maxTokens | Maximum response tokens | 4096 |

## Features

### Smart routing

LLMGateway can automatically route requests based on model availability, cost, or latency:

``` json
{
  "llmgateway": {
    "routing": {
      "strategy": "cost-optimized",
      "providers": ["openai", "anthropic", "google"]
    }
  }
}
```

### Response caching

Cache identical requests to reduce costs:

``` json
{
  "llmgateway": {
    "caching": true,
    "cacheTTL": 3600
  }
}
```

### Automatic failover

Automatically switch providers when one fails:

``` json
{
  "llmgateway": {
    "model": "gpt-4o",
    "fallbackModels": [
      "z-ai/glm-4.7",
      "gemini-1.5-pro"
    ]
  }
}
```

## Available models

Access any model from your configured upstream providers. Common options include:

| Provider | Models |
|---|---|
| OpenAI | gpt-4o, gpt-4-turbo, o1 |
| Anthropic | z-ai/glm-4.7, claude-3-opus |
| Google | gemini-1.5-pro, gemini-1.5-flash |
| Meta | llama-3.1-70b, llama-3.1-405b |

## Best practices

-   **Configure fallbacks**: Set up multiple fallback models for reliability.
-   **Enable caching**: Reduce costs for repeated queries.
-   **Set budgets**: Configure spending limits in the LLMGateway dashboard.
-   **Monitor usage**: Use the analytics dashboard to track costs and usage.

## Resources

-   [LLMGateway Website](https://llmgateway.io)
-   [Documentation](https://docs.llmgateway.io)
-   [Dashboard](https://app.llmgateway.io)
-   [API Reference](https://docs.llmgateway.io/api-reference)
-   [Pricing](https://llmgateway.io/pricing)

## Troubleshooting

### Common issues

| Issue | Solution |
|---|---|
| Invalid API key | Verify key at llmgateway.io |
| Upstream error | Check provider configuration in dashboard |
| Rate limited | Check rate limit settings |

### Debug mode

``` bash
# Enable verbose logging
AUTOHAND_DEBUG=true autohand --provider llmgateway
```