Primary Error Manifestation

All Google Generative AI models fail with identical, non-descriptive 400 errors regardless of model selection:

{
  "error": "Google Generative AI API error (400)",
  "rawErrorPreview": "Google Generative AI API error (400)",
  "model": "gemini-2.5-flash",
  "provider": "google"
}

Verified Working Components

• API Key Validity: Direct curl requests succeed with identical API key
• Auth Profile Configuration: Present and correctly structured
• Model Routing: Gateway confirms correct model selection

# Successful direct API call
$ curl "https://generativelanguage.googleapis.com/v1beta/models/gemini-2.5-flash:generateContent" \
  -H 'Content-Type: application/json' \
  -H 'X-goog-api-key: [REDACTED]' \
  -X POST \
  -d '{"contents":[{"parts":[{"text":"Say hi"}]}]}'

# Response: {"candidates":[{"content":{"parts":[{"text":"Hi there!"}]...}}

Failed OpenClaw Request Pattern

[gateway] agent model: google/gemini-2.5-flash
[gateway] ready
[google] Google Generative AI API error (400)
[google] rawErrorPreview: Google Generative AI API error (400)

Environment Details

• OpenClaw version: 2026.5.3-1
• OS: Ubuntu 24
• Install method: VPS with TailScale tunnel
• Tested models: google/gemini-2.5-flash, google/gemma-4-31b-it

Primary Suspect: Request Header Malformation

The critical diagnostic indicator is that the same API key succeeds via curl but fails in OpenClaw. This eliminates API key validity, network routing, and model availability as root causes, pointing definitively to request construction issues.

Potential Failure Vectors

1. Authorization Header Collision

   OpenClaw may be injecting an Authorization: Bearer header while Google API requires the proprietary X-goog-api-key header. Some HTTP clients concatenate headers incorrectly:

   # Incorrect - OpenClaw may be sending both:
   Authorization: Bearer [API_KEY]
   X-goog-api-key: [API_KEY]
   Google API rejects this with 400

2. Request Body Serialization Mismatch

   Google's generateContent endpoint has strict JSON schema requirements. OpenClaw may be:

   • Wrapping the request in an extra object layer
   • Using incorrect field names (e.g., prompt vs contents)
   • Serializing arrays incorrectly for the parts structure
3. API Endpoint Version Drift

   The working curl uses v1beta endpoint. OpenClaw may be calling v1 or an incorrect endpoint path:

   # Correct endpoint format:
   /v1beta/models/[MODEL_NAME]:generateContent
   Potential OpenClaw misconstruction:
   /v1/models/[MODEL_NAME]:generateContent
   /v1beta/generateContent  # Missing model name
   /v1/models/generateContent  # Wrong path structure

4. Content-Type or Accept Header Issues

   Missing Content-Type: application/json or incorrect Accept header can cause 400 errors.

5. TailScale Tunnel Interference

   Network-level interference could modify headers in transit, though this is unlikely given curl succeeds through the same path.

Architectural Context

Given the regression nature (worked before), this suggests a recent update to the Google provider's request construction logic introduced a breaking change in header injection, body serialization, or endpoint construction.

Immediate Diagnostic Steps

1. Enable debug logging to capture raw request:

   OPENCLAW_LOG_LEVEL=debug openclaw [command]

2. Compare request traces - Look for header differences between successful curl and OpenClaw request

Workaround: Use OpenRouter as Intermediary

If the Google provider cannot be fixed immediately, route through OpenRouter which handles API compatibility:

# Update openclaw configuration
openclaw config set agent.model "openrouter/google/gemini-2.5-flash"

# Or via environment variable
export OPENCLAW_MODEL="openrouter/google/gemini-2.5-flash"

Configuration Adjustment: Verify Auth Profile Structure

# Current (verify this matches):
openclaw config get auth.profiles
{
  "google:default": {
    "provider": "gemini",
    "mode": "api_key"
  }
}

# Alternative explicit configuration:
openclaw config set auth.profiles.google:default.provider "gemini"
openclaw config set auth.profiles.google:default.mode "api_key"
openclaw config set auth.profiles.google:default.api_key_env "GEMINI_API_KEY"

Environment File Verification

# Ensure ~/.openclaw/.env contains:
GEMINI_API_KEY=AIzaSy...

# Verify no conflicting variables:
grep -E "(GOOGLE_API_KEY|AIza)" ~/.openclaw/.env

Request Provider Override (if available)

# Try explicit provider specification:
openclaw config set agent.provider "google"
openclaw config set agent.model "gemini-2.5-flash"  # Without google/ prefix

Test 1: Direct Curl Verification (Baseline)

# Execute this exact request to establish baseline:
curl -s -w "\nHTTP_CODE:%{http_code}" \
  "https://generativelanguage.googleapis.com/v1beta/models/gemini-2.5-flash:generateContent" \
  -H 'Content-Type: application/json' \
  -H 'X-goog-api-key: YOUR_ACTUAL_KEY' \
  -X POST \
  -d '{"contents":[{"parts":[{"text":"test"}]}]}'

# Expected: HTTP_CODE:200 with JSON response

Test 2: OpenClaw Configuration Validation

# Verify auth profile exists and is correct:
openclaw config get auth.profiles.google:default
# Expected: {"provider": "gemini", "mode": "api_key"}

# Verify model mapping:
openclaw models list | grep -i google
# Expected: List of available google/* models

Test 3: Simple Completion Test

# Minimal test with debug output:
OPENCLAW_LOG_LEVEL=debug openclaw chat "Hi" --model google/gemini-2.5-flash 2>&1 | head -50

# Success indicators:
# - No "400" error in output
# - Contains "Hi" or response text
# - Exit code 0

Test 4: Alternative Model Verification

# Test with different model to isolate model-specific issues:
openclaw chat "test" --model google/gemini-1.5-flash

# Test Gemma specifically:
openclaw chat "test" --model google/gemma-4-31b-it

Test 5: OpenRouter Fallback Verification

# If using OpenRouter workaround:
openclaw chat "test" --model openrouter/google/gemini-2.5-flash

# Expected: Success with response from Gemini model

Expected Success Output

{
  "message": "Hi there! I'm doing well, thank you for asking...",
  "model": "gemini-2.5-flash",
  "provider": "google",
  "usage": {...}
}

Environment Variable Collision

• Multiple API key environment variables can cause provider confusion
• Check for conflicts: GOOGLE_API_KEY, GEMINI_API_KEY, OPENAI_API_KEY
• Ensure only the correct variable is set in ~/.openclaw/.env

# Wrong - multiple keys:
GEMINI_API_KEY=xxx
GOOGLE_API_KEY=yyy  # May override or conflict

# Correct - single key:
GEMINI_API_KEY=xxx

Model Name Format Inconsistencies

• Some versions require google/gemini-2.5-flash prefix
• Others require bare gemini-2.5-flash
• API endpoint differences: v1beta vs v1

TailScale/Network-Specific Issues

• DNS resolution may differ between local and VPS environments
• MTU/fragmentation issues with large requests
• Proxy headers may be added unintentionally

Version-Specific Provider Bugs

• Version 2026.5.3-1 exhibits this regression
• Check if downgrading resolves issues:

# Check current version:
openclaw --version

# If downgrade needed (caution - may affect other functionality):
# Consult documentation for version installation procedures

Auth Profile Priority

• Multiple overlapping auth profiles can cause unexpected behavior
• Explicitly specify profile: auth.profiles.google:default
• Clear orphaned profiles:

openclaw config unset auth.profiles.google:alternate 2>/dev/null
openclaw config get auth.profiles

API Quota and Regional Restrictions

• Google API keys can be restricted to specific domains/IPs
• VPS IP may not be whitelisted in Google Cloud Console
• Check Google Cloud Console → APIs & Services → Credentials

Authentication and Header Errors

• 401 Unauthorized - Invalid or missing API key, typically indicates key not being passed correctly
• 403 Forbidden - API key lacks permissions or is restricted to specific APIs
• 429 Too Many Requests - Rate limiting; different from 400 but may appear in logs

Request Format Errors

• 400: Invalid JSON - Request body serialization failure
• 400: Required field 'contents' not present - Incorrect request schema
• 400: Invalid model name - Model not found or not available for API key

Network and Connectivity Errors

• Connection timeout - Network routing issue, particularly relevant with TailScale
• SSL certificate verification failed - TLS/intermediate certificate issue
• DNS resolution failure - Cannot reach generativelanguage.googleapis.com

OpenClaw Internal Errors

• Provider not configured - Auth profile missing or misconfigured
• Model not found - Model name not in provider's supported list
• No valid auth credentials - API key not found in environment

Historical Related Issues

• OpenRouter provider works - Indicates issue is specifically with OpenClaw's Google provider implementation
• Curl succeeds - Confirms API endpoint, authentication, and network path are valid
• Regression pattern - Previously working Google integration now fails, pointing to recent code changes