vllm‑helper

Docs GitHub

Documentation

Introduction
Quickstart
Install and upgrade
Profiles
Models and quantizations
Memory budget
Web panel
Coder integration
Recipes
More than one machine
Running on macOS
Troubleshooting
CLI reference

vllm-helper pool

Run a router in front of replicas serving the same model

Usage

vllm-helper pool

← vllm-helper node rm vllm-helper pool install-service →

View this page as Markdown

On this page

Usage