Search...

Deployment with Helm

This guide explains how to deploy the Inference Gateway using the official Helm charts. If you'd rather manage gateways and related resources declaratively as Custom Resources, see the Kubernetes Operator.

Prerequisites

  • Kubernetes cluster (v1.30+)
  • Helm installed (v3.8+)
  • kubectl configured to access your cluster

Installation

Basic Gateway Deployment

Install the chart directly from OCI registry:

Terminal
helm upgrade --install inference-gateway oci://ghcr.io/inference-gateway/charts/inference-gateway \
  --namespace inference-gateway \
  --create-namespace \
  --version 0.23.1

To enable ingress for the basic deployment:

Terminal
helm upgrade --install inference-gateway oci://ghcr.io/inference-gateway/charts/inference-gateway \
  --namespace inference-gateway \
  --create-namespace \
  --set ingress.enabled=true \
  --version 0.23.1

TLS Certificate Management

Option 1: Manual TLS Secret Creation

For production deployments, you can manually create a TLS secret with your certificates:

Terminal
kubectl create secret tls inference-gateway-tls \
  --namespace inference-gateway \
  --cert=/path/to/tls.crt \
  --key=/path/to/tls.key

For production deployments, we recommend using cert-manager to automate certificate management. This approach automates certificate issuance and renewal:

  1. Install cert-manager if not already installed:
Terminal
helm repo add jetstack https://charts.jetstack.io
helm repo update
helm upgrade --install cert-manager jetstack/cert-manager \
  --namespace cert-manager \
  --create-namespace \
  --version v1.17.1 \
  --set crds.enabled=true
  1. Create an Issuer or ClusterIssuer (example with Let's Encrypt):
Terminal
kubectl apply -f - <<EOF
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-prod
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: [email protected]
    privateKeySecretRef:
      name: letsencrypt-prod
    solvers:
    - http01:
        ingress:
          class: nginx
EOF
  1. Update your Helm installation to use cert-manager annotations by adding the cert-manager.io/cluster-issuer annotation to your ingress configuration.

With this setup, cert-manager will automatically request and manage TLS certificates from Let's Encrypt.

Configuration

To see the available configurations, run:

Terminal
helm show values oci://ghcr.io/inference-gateway/charts/inference-gateway

Upgrading

To upgrade an existing installation:

Terminal
helm upgrade --install inference-gateway oci://ghcr.io/inference-gateway/charts/inference-gateway \
  --namespace inference-gateway \
  -f values.yaml \
  --version 0.23.1

Uninstalling

To completely remove the installation:

Terminal
helm uninstall inference-gateway --namespace inference-gateway

# Remove the namespace
kubectl delete namespace inference-gateway

Checking Deployment Status

Terminal
# Check running pods
kubectl get pods -n inference-gateway

# Check services
kubectl get svc -n inference-gateway

# Check ingress (if enabled)
kubectl get ingress -n inference-gateway

Chart Sources

The Helm chart sources are available at: