Inference Gateway CLI

The Inference Gateway CLI (infer) is a powerful Go-based command-line tool that provides comprehensive access to the Inference Gateway. It features interactive chat, autonomous agent capabilities, extensive tool integration, and advanced conversation management.

Current Version: v0.47.0 (Breaking changes expected until stable)

Installation

Install Script (Recommended)

Terminal

# Latest version
curl -fsSL https://raw.githubusercontent.com/inference-gateway/cli/main/install.sh | bash

Terminal

# Specific version
curl -fsSL https://raw.githubusercontent.com/inference-gateway/cli/main/install.sh | bash -s -- --version v0.47.0

Terminal

# Custom directory
curl -fsSL https://raw.githubusercontent.com/inference-gateway/cli/main/install.sh | bash -s -- --install-dir $HOME/.local/bin

Go Install

If you have Go installed:

Terminal

go install github.com/inference-gateway/cli@latest

Manual Download

Download binaries from the GitHub releases page. Binaries are signed with Cosign for verification.

Build from Source

Terminal

git clone https://github.com/inference-gateway/cli.git
cd cli
go build -o infer

Quick Start

Inference Gateway TUI Interface

Initialize your project and start using the CLI:

Terminal

# Initialize configuration
infer init

# Check gateway status
infer status

# Start interactive chat
infer chat

# Get help
infer --help

Core Commands

Essential Commands

`infer init`

Initialize your project with .infer directory and configuration:

Terminal

infer init

Creates .infer/config.yaml with default settings and tool configurations.

`infer status`

Check gateway health and resource usage:

Terminal

infer status

`infer chat`

Launch interactive chat with TUI interface:

Terminal

infer chat

Features scrolling, keyboard navigation, model selection, and tool result expansion.

`infer agent`

Autonomous agent mode - Execute complex tasks in the background:

Terminal

infer agent "Analyze this codebase and suggest improvements"
infer agent "Fix the failing tests in the test suite"
infer agent "Implement a new feature based on issue #123"

The agent operates autonomously with task analysis, planning, execution, and validation phases.

Configuration Management

Initialize Configuration Only

Terminal

infer config init

Agent Configuration

Terminal

# Set default model for chat
infer config agent set-model openai/gpt-4

# Set system prompt
infer config agent set-system "You are a helpful coding assistant"

Tool Management

Terminal

# Enable/disable tool execution
infer config tools enable
infer config tools disable

# Manage command whitelist
infer config tools list
infer config tools validate
infer config tools exec

# Safety settings
infer config tools safety enable    # Require approval prompts
infer config tools safety disable
infer config tools safety status

# Sandbox management
infer config tools sandbox add /protected/path
infer config tools sandbox remove /protected/path
infer config tools sandbox list

Configuration

The CLI uses a 2-layer configuration system with precedence:

Environment Variables (INFER_* prefix) - Highest priority
Command Line Flags
Project Config (.infer/config.yaml)
User Config (~/.infer/config.yaml)
Built-in Defaults - Lowest priority

Configuration File Structure

The CLI creates a comprehensive YAML configuration:

YAML

gateway:
  url: http://localhost:8080
  api_key: ""
  timeout: 200
client:
  timeout: 200
  retry:
    enabled: true
    max_attempts: 3
    initial_backoff_sec: 5
    max_backoff_sec: 60
    backoff_multiplier: 2
    retryable_status_codes: [400, 408, 429, 500, 502, 503, 504]
logging:
  debug: false
tools:
  enabled: true # Tools are enabled by default with safe read-only commands
  sandbox:
    directories: [".", "/tmp"] # Allowed directories for tool operations
    protected_paths: # Paths excluded from tool access for security
      - .infer/
      - .git/
      - *.env
  bash:
    enabled: true
    whitelist:
      commands: # Exact command matches
        - ls
        - pwd
        - echo
        - wc
        - sort
        - uniq
        - gh
        - task
      patterns: # Regex patterns for more complex commands
        - ^git branch( --show-current)?$
        - ^git checkout -b [a-zA-Z0-9/_-]+( [a-zA-Z0-9/_-]+)?$
        - ^git checkout [a-zA-Z0-9/_-]+
        - ^git add [a-zA-Z0-9/_.-]+
        - ^git diff+
        - ^git remote -v$
        - ^git status$
        - ^git log --oneline -n [0-9]+$
        - ^git commit -m ".+"$
        - ^git push( --set-upstream)?( origin)?( [a-zA-Z0-9/_-]+)?$
  read:
    enabled: true
    require_approval: false
  write:
    enabled: true
    require_approval: true # Write operations require approval by default for security
  edit:
    enabled: true
    require_approval: true # Edit operations require approval by default for security
  delete:
    enabled: true
    require_approval: true # Delete operations require approval by default for security
  grep:
    enabled: true
    backend: auto # "auto", "ripgrep", or "go"
    require_approval: false
  tree:
    enabled: true
    require_approval: false
  web_fetch:
    enabled: true
    whitelisted_domains:
      - golang.org
    safety:
      max_size: 4096 # 4KB
      timeout: 30 # 30 seconds
      allow_redirect: true
    cache:
      enabled: true
      ttl: 3600 # 1 hour
      max_size: 52428800 # 50MB
  web_search:
    enabled: true
    default_engine: duckduckgo
    max_results: 10
    engines:
      - duckduckgo
      - google
    timeout: 10
  todo_write:
    enabled: true
    require_approval: false
  github:
    enabled: true
    token: "%GITHUB_TOKEN%"
    base_url: "https://api.github.com"
    owner: ""
    safety:
      max_size: 1048576  # 1MB
      timeout: 30        # 30 seconds
    require_approval: false
  safety:
    require_approval: true
compact:
  output_dir: .infer # Directory for compact command exports
  summary_model: "" # Model to use for summarization (optional)
agent:
  model: "" # Default model for agent operations
  system_prompt: |
    System prompt for the main agent
  system_reminders:
    enabled: true
    interval: 4
    reminder_text: |
      System reminder text for maintaining context
  verbose_tools: false
  max_turns: 50 # Maximum number of turns for agent sessions
  max_tokens: 4096 # The maximum number of tokens that can be generated per request
  optimization:
    enabled: false
    max_history: 10
    compact_threshold: 20
    truncate_large_outputs: true
    skip_redundant_confirmations: true
chat:
  theme: tokyo-night

Environment Variables

Terminal

export INFER_GATEWAY_URL="http://localhost:3000"
export INFER_GATEWAY_API_KEY="your-api-key"
export INFER_AGENT_MODEL="openai/gpt-4"
export INFER_LOGGING_DEBUG="true"

Advanced Features

Tool System for LLMs

When enabled, LLMs have access to a comprehensive tool suite:

File System Tools

Bash: Execute whitelisted shell commands
Read/Write/Edit: File operations with safety controls
MultiEdit: Batch file edits
Delete/Tree: File management and exploration

Search Tools

Grep: Powered by ripgrep for fast code search
WebSearch/WebFetch: Internet research capabilities

Development Tools

GitHub API: Repository integration
TodoWrite: Task management for complex workflows

Security Features

Command Whitelisting: Only approved patterns allowed
Approval Prompts: Safety confirmations for dangerous operations
Path Protection: Sensitive directories automatically excluded
Sandbox Controls: Protected directory management

Conversation Management

Storage Backends

SQLite (default): Local file-based storage
PostgreSQL: Shared database for teams
Redis: High-performance caching

Features

Automatic conversation history with search
Intelligent title generation
Token optimization and compaction
Export/import capabilities

Interactive Interface

The TUI provides:

Scrollable conversation view
Keyboard shortcuts for navigation
Tool result expansion/collapse
Real-time streaming responses
Model switching during conversation

Git Shortcuts

/git <command> [args...] - Execute git commands (supports commit, push, status, etc.)
/git commit [flags] - NEW: Commit staged changes with AI-generated message
/git push [remote] [branch] [flags] - NEW: Push commits to remote repository

The git shortcuts provide intelligent commit message generation using AI when no message is provided with /git commit.

User-Defined Shortcuts

You can create custom shortcuts by adding YAML configuration files in the .infer/shortcuts/ directory.

Configuration File Format

Create files named custom-*.yaml (e.g., custom-1.yaml, custom-dev.yaml) in .infer/shortcuts/:

YAML

shortcuts:
  - name: 'tests'
    description: 'Run all tests in the project'
    command: 'go'
    args: ['test', './...']
    working_dir: '.' # Optional: set working directory

  - name: 'build'
    description: 'Build the project'
    command: 'go'
    args: ['build', '-o', 'infer', '.']

  - name: 'lint'
    description: 'Run linter on the codebase'
    command: 'golangci-lint'
    args: ['run']

Security & Safety

Command Whitelisting

Terminal

# Add allowed commands
infer config tools whitelist add "npm install"
infer config tools whitelist add "git log --oneline"

# Remove from whitelist
infer config tools whitelist remove "dangerous-command"

Protected Paths

Sensitive directories are automatically protected:

.git/ - Git repository data
*.env - Environment files
node_modules/ - Dependencies
Custom paths via sandbox configuration

Approval Prompts

Enable safety confirmations:

Terminal

infer config tools safety enable

LLMs will request approval before executing potentially dangerous operations.

Integration Examples

Shell Aliases

Terminal

# Add to .bashrc or .zshrc
alias ask="infer chat"
alias ai-agent="infer agent"
alias ai-status="infer status"

Development Workflow

Terminal

# Initialize new project
infer init

# Agent-driven development
infer agent "Implement user authentication with JWT"

# Interactive problem solving
infer chat
> How do I optimize this database query?

CI/CD Integration

Terminal

#!/bin/bash
# Automated code review
infer agent "Review the changes in this PR and suggest improvements"

Troubleshooting

Connection Issues

Terminal

# Check configuration
infer config show

# Verify gateway status
infer status

# Debug mode
infer --debug chat

Permission Issues

Terminal

# Check configuration directory
ls -la ~/.infer/

# Reset configuration
infer config reset

# Re-initialize
infer init

Tool Execution Problems

Terminal

# Check tool status
infer config tools status

# Validate whitelist
infer config tools validate

# Enable debug logging
export INFER_DEBUG=true
infer agent "your task"

Command Reference

Command	Description
`infer init`	Initialize project configuration
`infer status`	Check gateway health
`infer chat`	Interactive chat session
`infer agent <task>`	Autonomous task execution
`infer config <subcommand>`	Configuration management
`infer --version`	Show version information
`infer --help`	Display help information

Support and Resources

Repository: github.com/inference-gateway/cli
Issues: GitHub Issues
Releases: GitHub Releases

The CLI is actively developed with regular updates and new features. Check the repository for the latest releases and announcements.