---
title: "GCP Vertex AI Integration"
source: https://docs.autohand.ai/integrations/gcp-vertex
---

# GCP Vertex AI

Access Google's Gemini models and open-source models through Vertex AI. Enterprise-grade infrastructure with Google Cloud security and compliance.

## Overview

Google Cloud Vertex AI provides enterprise access to AI models with Google Cloud infrastructure. When integrated with Autohand, you get:

-   Access to Gemini 1.5 Pro, Gemini 1.5 Flash, and Gemini 2.0
-   Open-source models like Llama 3.1 and Mistral through Model Garden
-   Up to 2 million token context window with Gemini
-   Enterprise security with Google Cloud IAM
-   Regional endpoints for data residency
-   Integration with existing GCP infrastructure

**Long context:** Gemini 1.5 Pro offers a 2 million token context window, making it ideal for analyzing large codebases or documentation.

## Setup

Get started with GCP Vertex AI.

### Prerequisites

-   Google Cloud project with Vertex AI API enabled
-   Service account with Vertex AI User role
-   gcloud CLI installed and configured

### Enable Vertex AI

``` bash
# Enable the Vertex AI API
gcloud services enable aiplatform.googleapis.com

# Set your project
gcloud config set project YOUR_PROJECT_ID
```

### Configure authentication

``` bash
# Option 1: Application Default Credentials (recommended for development)
gcloud auth application-default login

# Option 2: Service account key
export GOOGLE_APPLICATION_CREDENTIALS="/path/to/service-account.json"
```

### Configure Autohand

``` bash
# Set environment variables
export GOOGLE_CLOUD_PROJECT="your-project-id"
export GOOGLE_CLOUD_REGION="us-central1"

# Or configure via CLI
autohand config set vertex.project "your-project-id"
autohand config set vertex.region "us-central1"
```

Verify your configuration:

``` bash
# Start with Vertex AI provider
autohand --provider vertex --model gemini-1.5-pro

# Test with a prompt
autohand --prompt "Hello, which model are you?"
```

## CLI configuration

Configure Vertex AI in your `~/.autohand/config.json`:

``` json
{
  "provider": "vertex",
  "vertex": {
    "project": "${GOOGLE_CLOUD_PROJECT}",
    "region": "us-central1",
    "model": "gemini-1.5-pro",
    "maxTokens": 8192,
    "temperature": 0.7
  }
}
```

### Configuration options

| Option | Description | Default |
|---|---|---|
| project | Google Cloud project ID | - |
| region | GCP region for Vertex AI | us-central1 |
| model | Model to use | gemini-1.5-pro |
| maxTokens | Maximum response tokens | 8192 |
| temperature | Response randomness | 0.7 |
| credentials | Path to service account JSON | ADC |

## Available models

Vertex AI offers Google's Gemini models and open-source alternatives.

### Gemini models

| Model | Context | Best for |
|---|---|---|
| gemini-2.0-flash | 1M | Fastest, multimodal |
| gemini-1.5-pro | 2M | Long context, complex tasks |
| gemini-1.5-flash | 1M | Fast and cost-effective |

### Model Garden (open-source)

| Model | Context | Best for |
|---|---|---|
| llama-3.1-70b | 128K | Open-source, flexible |
| mistral-large | 128K | Efficient reasoning |
| codellama-34b | 16K | Code generation |

### Switch models

``` bash
# Set default model
autohand config set model "gemini-1.5-pro"

# Use during a session
/model vertex/gemini-2.0-flash
```

## Best practices

-   **Use ADC in development**: Application Default Credentials are easier to manage locally.
-   **Use service accounts in production**: Create dedicated service accounts with minimal permissions.
-   **Choose the right region**: Deploy in regions close to your users for lower latency.
-   **Monitor costs**: Use Google Cloud Billing to track AI spending.
-   **Use Gemini 1.5 Pro for large context**: Ideal for analyzing entire codebases.

### Recommended configuration

``` json
{
  "vertex": {
    "model": "gemini-1.5-pro",
    "temperature": 0.3,
    "maxTokens": 8192
  }
}
```

## Resources

-   [Vertex AI Overview](https://cloud.google.com/vertex-ai)
-   [Vertex AI Documentation](https://cloud.google.com/vertex-ai/docs)
-   [Generative AI on Vertex AI](https://cloud.google.com/vertex-ai/generative-ai/docs/learn/overview)
-   [Vertex AI Console](https://console.cloud.google.com/vertex-ai)
-   [Pricing](https://cloud.google.com/vertex-ai/pricing)
-   [Google Cloud Status](https://status.cloud.google.com)

## Troubleshooting

### Common issues

| Issue | Solution |
|---|---|
| Permission denied | Grant Vertex AI User role to service account |
| API not enabled | Run gcloud services enable aiplatform.googleapis.com |
| Region not supported | Use a supported region like us-central1 |
| Quota exceeded | Request quota increase in GCP Console |

### Debug mode

``` bash
# Enable verbose logging
AUTOHAND_DEBUG=true autohand --provider vertex

# Test authentication
gcloud auth application-default print-access-token
```