Examples

This page provides examples of how to use Inference Gateway in various scenarios and environments.

Docker Compose Examples

Docker Compose provides a simple way to set up Inference Gateway with various configurations.

Basic Setup

A minimal setup with OpenAI integration:

YAML

version: '3'
services:
  inference-gateway:
    image: ghcr.io/inference-gateway/inference-gateway:latest
    environment:
      - OPENAI_API_KEY=your-api-key
    ports:
      - '8080:8080'

For more Docker Compose examples, check the official examples repository.

Kubernetes Examples

Deploy Inference Gateway on Kubernetes with these example configurations.

YAML

apiVersion: apps/v1
kind: Deployment
metadata:
  name: inference-gateway
spec:
  replicas: 2
  selector:
    matchLabels:
      app: inference-gateway
  template:
    metadata:
      labels:
        app: inference-gateway
    spec:
      containers:
        - name: inference-gateway
          image: ghcr.io/inference-gateway/inference-gateway:latest
          ports:
            - containerPort: 8080
          env:
            - name: OPENAI_API_KEY
              valueFrom:
                secretKeyRef:
                  name: llm-secrets
                  key: openai-api-key

For complete Kubernetes deployment examples including Services, ConfigMaps, and Secrets, visit the Kubernetes examples in the GitHub repository.

API Usage Examples

Create Chat Completions with OpenAI

Terminal

curl -X POST http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-4",
    "messages": [
      {
        "role": "system",
        "content": "You are a helpful assistant."
      },
      {
        "role": "user",
        "content": "Explain how Inference Gateway works."
      }
    ]
  }'

Create Chat Completions with Anthropic

Terminal

curl -X POST http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "anthropic/claude-3-opus-20240229",
    "messages": [
      {
        "role": "system",
        "content": "You are a helpful assistant."
      },
      {
        "role": "user",
        "content": "Compare and contrast different LLM providers."
      }
    ]
  }'

For more detailed examples and use cases, check out the full examples directory in the GitHub repository.