Inference Gateway

Appearance

Getting Started ​

Learn how to install and set up Inference Gateway.

Installation ​

Using Docker ​

bash
docker pull ghcr.io/inference-gateway/inference-gateway:latest
docker run --rm -it -p 8080:8080 -e OPENAI_API_KEY=your_key_here ghcr.io/inference-gateway/inference-gateway:latest

Using Docker Compose ​

Checkout the examples in the Docker Compose examples.

Using Kubernetes ​

Deploy to Kubernetes with the Kubernetes Operator - the recommended path. It manages the gateway and related resources declaratively as Custom Resources. Follow the Operator quick start to apply your first Gateway, or browse the runnable Kubernetes examples.

Basic Usage ​

Send a request to the Inference Gateway:

bash
curl -X POST http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-5.6-luna",
    "messages": [
      {
        "role": "system",
        "content": "You are a helpful assistant."
      },
      {
        "role": "user",
        "content": "Hello, world!"
      }
    ]
  }'

The model id is always <provider>/<model>, so switching providers is a one-word change: set ANTHROPIC_API_KEY in the Docker step and ask for anthropic/claude-opus-4-8, or set GOOGLE_API_KEY and ask for google/gemini-3-flash. The gateway returns 400 Bad Request if the provider in the model id has no API key configured.

Next steps ​