Search...

Deployment with Helm

This guide explains how to deploy the Inference Gateway using the official Helm charts.

Prerequisites

  • Kubernetes cluster (v1.30+)
  • Helm installed (v3.8+)
  • kubectl configured to access your cluster

Installation

Basic Gateway Deployment

Install the chart directly from OCI registry:

Terminal
helm upgrade --install inference-gateway oci://ghcr.io/inference-gateway/charts/inference-gateway \
  --namespace inference-gateway \
  --create-namespace \
  --version 0.23.1

To enable ingress for the basic deployment:

Terminal
helm upgrade --install inference-gateway oci://ghcr.io/inference-gateway/charts/inference-gateway \
  --namespace inference-gateway \
  --create-namespace \
  --set ingress.enabled=true \
  --version 0.23.1

TLS Certificate Management

Option 1: Manual TLS Secret Creation

For production deployments, you can manually create a TLS secret with your certificates:

Terminal
kubectl create secret tls inference-gateway-tls \
  --namespace inference-gateway \
  --cert=/path/to/tls.crt \
  --key=/path/to/tls.key

For production deployments, we recommend using cert-manager to automate certificate management. This approach automates certificate issuance and renewal:

  1. Install cert-manager if not already installed:
Terminal
helm repo add jetstack https://charts.jetstack.io
helm repo update
helm upgrade --install cert-manager jetstack/cert-manager \
  --namespace cert-manager \
  --create-namespace \
  --version v1.17.1 \
  --set crds.enabled=true
  1. Create an Issuer or ClusterIssuer (example with Let's Encrypt):
Terminal
kubectl apply -f - <<EOF
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-prod
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: [email protected]
    privateKeySecretRef:
      name: letsencrypt-prod
    solvers:
    - http01:
        ingress:
          class: nginx
EOF
  1. Update your Helm installation to use cert-manager annotations by adding the cert-manager.io/cluster-issuer annotation to your ingress configuration.

With this setup, cert-manager will automatically request and manage TLS certificates from Let's Encrypt.

Configuration

To see the available configurations, run:

Terminal
helm show values oci://ghcr.io/inference-gateway/charts/inference-gateway

Upgrading

To upgrade an existing installation:

Terminal
helm upgrade --install inference-gateway oci://ghcr.io/inference-gateway/charts/inference-gateway \
  --namespace inference-gateway \
  -f values.yaml \
  --version 0.23.1

Uninstalling

To completely remove the installation:

Terminal
helm uninstall inference-gateway --namespace inference-gateway

# Remove the namespace
kubectl delete namespace inference-gateway

Checking Deployment Status

Terminal
# Check running pods
kubectl get pods -n inference-gateway

# Check services
kubectl get svc -n inference-gateway

# Check ingress (if enabled)
kubectl get ingress -n inference-gateway

Chart Sources

The Helm chart sources are available at: