User Interface (UI)
Inference Gateway includes an optional web-based user interface for interactively testing and exploring the various language model providers and capabilities.
Overview
The Inference Gateway UI provides a user-friendly interface for interacting with various language model providers. It allows users to test different models, experiment with parameters, and view the responses in real-time without writing any code.
Installation
The UI is a separate application that uses the Inference Gateway installation as a backend.
Model Selection
Once you've selected a provider, you can choose from any of the available models offered by that provider. The UI automatically filters models based on the selected provider.
Chat Interface
The main interface provides a familiar chat experience:
- System prompt configuration
- Input area for new messages
- Support for markdown and code formatting in responses
- Streaming responses for a more interactive experience
- Reasoning and thought process visualization
- Chat usage statistics
Screenshots
The UI features a clean, modern design with light and dark mode support. The layout is responsive and works well on desktop and mobile devices.
Configuration
The UI can be configured through environment variables:
INFERENCE_GATEWAY_URL: The URL of the Inference Gateway server (default:http://localhost:8080/v1).
Development
The UI is built with modern web technologies:
- NextJS with React for the component structure
- TypeScript for type safety
- Tailwind CSS for styling
Deployment
The UI can be deployed as a standalone application. It provides a container image that could be deployed on any container orchestration platform, such as Kubernetes.
You can self host the UI by building it from source or using the provided Docker image. The UI is designed to be lightweight (approx. ~300MB) and can run on any server with Node.js installed.
Examples
For extensive examples checkout: github repository.
To customize or extend the UI, you can find the source code in the github repository.