llm-helper up
Start serving a profile alongside anything already running
Usage
llm-helper up <profile> [flags]
Options
--kv-cache
| Type | string |
fixed KV cache size in bytes, e.g. 20G
--mem-util
| Type | string |
fraction of GPU memory this model may use, e.g. 0.4
--no-follow
| Type | bool |
return as soon as the server is started instead of watching the boot
--port
| Type | string |
port to serve on (default: the next free one from PORT)
--timeout
| Type | duration |
give up waiting after this long (default: wait forever)
--wait
| Type | bool |
block until the endpoint is healthy, without streaming logs