vllm-helper shard start

Start a sharded vLLM server in the foreground

Usage

vllm-helper shard start --profile <profile> --nodes n1,n2 [flags]

Options

--backend

distributed executor backend: ray or mp

--distributed-timeout-seconds

vLLM distributed init timeout (0 uses vLLM default)

--dry-run

print commands instead of running them

--head

head node name (defaults to first --nodes entry)

--head-ip

IP workers use to reach the distributed head

--iface

network interface for distributed traffic

--install-ray

install Ray into the vLLM virtualenv before starting

--master-port

vLLM multiprocessing master port

--nodes

comma-separated node names

--port

OpenAI API port (defaults to config PORT)

--pp

pipeline parallel size

--profile

profile to serve

--ray-port

ray head port

--service-name

vLLM service to stop when --stop-services is set

--ssh-key

SSH private key for non-head nodes

--stop-services

stop existing vLLM services on all nodes before starting

--tp

tensor parallel size