---
# Code generated by go run ./scripts/gendocs. DO NOT EDIT.
title: "vllm-helper shard start"
description: "Start a sharded vLLM server in the foreground"
---

<!-- DO NOT EDIT | GENERATED CONTENT -->

## Usage

```console
vllm-helper shard start --profile <profile> --nodes n1,n2 [flags]
```

## Options

### --backend

|         |          |
| ------- | -------- |
| Type    | `string` |
| Default | `ray`    |

distributed executor backend: ray or mp

### --distributed-timeout-seconds

|      |       |
| ---- | ----- |
| Type | `int` |

vLLM distributed init timeout (0 uses vLLM default)

### --dry-run

|      |        |
| ---- | ------ |
| Type | `bool` |

print commands instead of running them

### --head

|      |          |
| ---- | -------- |
| Type | `string` |

head node name (defaults to first --nodes entry)

### --head-ip

|      |          |
| ---- | -------- |
| Type | `string` |

IP workers use to reach the distributed head

### --iface

|      |          |
| ---- | -------- |
| Type | `string` |

network interface for distributed traffic

### --install-ray

|      |        |
| ---- | ------ |
| Type | `bool` |

install Ray into the vLLM virtualenv before starting

### --master-port

|         |         |
| ------- | ------- |
| Type    | `int`   |
| Default | `29501` |

vLLM multiprocessing master port

### --nodes

|      |          |
| ---- | -------- |
| Type | `string` |

comma-separated node names

### --port

|      |          |
| ---- | -------- |
| Type | `string` |

OpenAI API port (defaults to config PORT)

### --pp

|         |       |
| ------- | ----- |
| Type    | `int` |
| Default | `2`   |

pipeline parallel size

### --profile

|      |          |
| ---- | -------- |
| Type | `string` |

profile to serve

### --ray-port

|         |        |
| ------- | ------ |
| Type    | `int`  |
| Default | `6379` |

ray head port

### --service-name

|         |                |
| ------- | -------------- |
| Type    | `string`       |
| Default | `vllm.service` |

vLLM service to stop when --stop-services is set

### --ssh-key

|      |          |
| ---- | -------- |
| Type | `string` |

SSH private key for non-head nodes

### --stop-services

|      |        |
| ---- | ------ |
| Type | `bool` |

stop existing vLLM services on all nodes before starting

### --tp

|         |       |
| ------- | ----- |
| Type    | `int` |
| Default | `1`   |

tensor parallel size