title: the summary lines of the sweep with prefill on the full card and decode on the shared card
source: manuals/sglang-101/capture/gpu/incidents/l4-pd-summary-lines.txt
lines: 2-8
lang: text
note: Follow the retokenized tokens and the throughput as the load grows.
---
load/pd/c1-s1: Successful requests: 16 |Total generated tokens: 2048 |Total generated tokens (retokenized): 2048 |Output token throughput (tok/s): 5.60 |
load/pd/c2-s1: Successful requests: 16 |Total generated tokens: 2048 |Total generated tokens (retokenized): 512 |Output token throughput (tok/s): 71.70 |
load/pd/c4-s1: Successful requests: 16 |Total generated tokens: 2048 |Total generated tokens (retokenized): 0 |Output token throughput (tok/s): 2516.97 |
rate/pd/r0.5-s1: Successful requests: 32 |Total generated tokens: 4096 |Total generated tokens (retokenized): 896 |Output token throughput (tok/s): 62.42 |
rate/pd/r0.5-s2: Successful requests: 32 |Total generated tokens: 4096 |Total generated tokens (retokenized): 0 |Output token throughput (tok/s): 113.28 |
rate/pd/r1-s1: Successful requests: 32 |Total generated tokens: 4096 |Total generated tokens (retokenized): 0 |Output token throughput (tok/s): 161.76 |
rate/pd/r8-s1: Successful requests: 240 |Total generated tokens: 30720 |Total generated tokens (retokenized): 0 |Output token throughput (tok/s): 751.85 |
