Inference Gateway

Appearance

Getting Started ​

Learn how to install and set up Inference Gateway.

Installation ​

Using Docker ​

bash
docker pull ghcr.io/inference-gateway/inference-gateway:latest
docker run --rm -it -p 8080:8080 -e OPENAI_API_KEY=your_key_here ghcr.io/inference-gateway/inference-gateway:latest

Using Docker Compose ​

Checkout the examples in the Docker Compose examples.

Using Kubernetes ​

Deploy to Kubernetes with the Kubernetes Operator - the recommended path. It manages the gateway and related resources declaratively as Custom Resources. Follow the Operator quick start to apply your first Gateway, or browse the runnable Kubernetes examples.

Basic Usage ​

Send a request to the Inference Gateway:

bash
curl -X POST http://localhost:8080/v1/chat/completions \
-d '{
"model": "deepseek/deepseek-v4-flash",
"messages": [
        {
            "role": "system",
            "content": "You are a helpful assistant."
        },
        {
            "role": "user",
            "content": "Hello, world!"
        }
    ]
}

Next steps ​