vllm-helper serve

Run a web panel for this machine

Serve a control panel: what model is loaded, whether it is healthy, and what else could be loaded instead.

It listens on localhost unless told otherwise. The panel can restart the server and change which model runs, so every request needs the token printed below -- open the URL it gives you.

The panel drives the native service (systemd or launchd) only: recipe container clusters are not controlled from it yet.

Usage

vllm-helper serve [flags]

Options

--listen

address to listen on (use 0.0.0.0:8080 to reach it from another machine)