vllm-helper pool serve
Run a sticky reverse proxy over the given nodes
Usage
vllm-helper pool serve --nodes n1,n2 [--bind :8011] [--sticky auto|authorization|header:Name] [flags]
Options
--bind
| Type | string |
| Default | :8011 |
Bind address for the router
--nodes
| Type | string |
Comma-separated node names (from 'node ls')
--sticky
| Type | string |
| Default | auto |
Stickiness (auto|authorization|header:Name)