vllm‑helper

Docs GitHub

Documentation

Introduction
Quickstart
Install and upgrade
Profiles
Models and quantizations
Memory budget
Web panel
Coder integration
Recipes
More than one machine
Running on macOS
Troubleshooting
CLI reference

vllm-helper shard

Split one model across machines (Ray or vLLM multiprocessing)

Usage

vllm-helper shard

← vllm-helper setup vllm-helper shard install-ray →

View this page as Markdown

On this page

Usage