title: the default start on the same card: the pool, the free memory and the prefill capture
source: manuals/sglang-101/capture/gpu/out-l4/server-default.log
lines: 19-23
lang: text
note: Read the available memory after the pool, the token count of the first captured shape, and the memory the capture used.
---
[2026-10-17 14:42:45] KV Cache is allocated. dtype: torch.bfloat16, #tokens: 22939, K size: 1.58 GB, V size: 1.58 GB
[2026-10-17 14:42:45] Memory pool end. avail mem=2.86 GB
[2026-10-17 14:42:45] Capture target prefill CUDA graph begin. backend=breakable, num_tokens=[4, 8, 12, 16, 20, 24, 28, 32, 48, 64, 80, 96, 112, 128, 144, 160, 176, 192, 208, 224, 240, 256, 288, 320, 352, 384, 416, 448, 480, 512, 576, 640, 704, 768, 832, 896, 960, 1024, 1280, 1536, 1792, 2048, 2304, 2560, 2816, 3072, 3328, 3584, 3840, 4096], avail mem=2.28 GB

  0%|          | 0/50 [00:00<?, ?it/s]
Capturing prefill shape (num_tokens=4096 avail_mem=2.28 GB):   0%|          | 0/50 [00:00<?, ?it/s]
Capturing prefill shape (num_tokens=4096 avail_mem=2.28 GB):   2%|▏         | 1/50 [00:02<02:13,  2.73s/it]
Capturing prefill shape (num_tokens=3840 avail_mem=1.28 GB):   2%|▏         | 1/50 [00:02<02:13,  2.73s/it]
Capturing prefill shape (num_tokens=3840 avail_mem=1.28 GB):   4%|▍         | 2/50 [00:07<03:16,  4.09s/it]
Capturing prefill shape (num_tokens=3584 avail_mem=1.25 GB):   4%|▍         | 2/50 [00:07<03:16,  4.09s/it]
Capturing prefill shape (num_tokens=3584 avail_mem=1.25 GB):   6%|▌         | 3/50 [00:09<02:20,  2.98s/it]
Capturing prefill shape (num_tokens=3328 avail_mem=1.25 GB):   6%|▌         | 3/50 [00:09<02:20,  2.98s/it]
Capturing prefill shape (num_tokens=3328 avail_mem=1.25 GB):   8%|▊         | 4/50 [00:14<02:50,  3.72s/it]
Capturing prefill shape (num_tokens=3072 avail_mem=1.24 GB):   8%|▊         | 4/50 [00:14<02:50,  3.72s/it]
Capturing prefill shape (num_tokens=3072 avail_mem=1.24 GB):  10%|█         | 5/50 [00:15<02:13,  2.97s/it]
Capturing prefill shape (num_tokens=2816 avail_mem=1.23 GB):  10%|█         | 5/50 [00:15<02:13,  2.97s/it]
Capturing prefill shape (num_tokens=2816 avail_mem=1.23 GB):  12%|█▏        | 6/50 [00:20<02:32,  3.47s/it]
Capturing prefill shape (num_tokens=2560 avail_mem=1.23 GB):  12%|█▏        | 6/50 [00:20<02:32,  3.47s/it]
Capturing prefill shape (num_tokens=2560 avail_mem=1.23 GB):  14%|█▍        | 7/50 [00:23<02:18,  3.23s/it]
Capturing prefill shape (num_tokens=2304 avail_mem=1.22 GB):  14%|█▍        | 7/50 [00:23<02:18,  3.23s/it]
Capturing prefill shape (num_tokens=2304 avail_mem=1.22 GB):  16%|█▌        | 8/50 [00:24<01:48,  2.59s/it]
Capturing prefill shape (num_tokens=2048 avail_mem=1.22 GB):  16%|█▌        | 8/50 [00:24<01:48,  2.59s/it]
Capturing prefill shape (num_tokens=2048 avail_mem=1.22 GB):  18%|█▊        | 9/50 [00:27<01:59,  2.91s/it]
Capturing prefill shape (num_tokens=1792 avail_mem=1.21 GB):  18%|█▊        | 9/50 [00:27<01:59,  2.91s/it]
Capturing prefill shape (num_tokens=1792 avail_mem=1.21 GB):  20%|██        | 10/50 [00:28<01:33,  2.34s/it]
Capturing prefill shape (num_tokens=1536 avail_mem=1.21 GB):  20%|██        | 10/50 [00:28<01:33,  2.34s/it]
Capturing prefill shape (num_tokens=1536 avail_mem=1.21 GB):  22%|██▏       | 11/50 [00:31<01:30,  2.32s/it]
Capturing prefill shape (num_tokens=1280 avail_mem=1.20 GB):  22%|██▏       | 11/50 [00:31<01:30,  2.32s/it]
Capturing prefill shape (num_tokens=1280 avail_mem=1.20 GB):  24%|██▍       | 12/50 [00:32<01:14,  1.97s/it]
Capturing prefill shape (num_tokens=1024 avail_mem=1.19 GB):  24%|██▍       | 12/50 [00:32<01:14,  1.97s/it]
Capturing prefill shape (num_tokens=1024 avail_mem=1.19 GB):  26%|██▌       | 13/50 [00:33<00:57,  1.56s/it]
Capturing prefill shape (num_tokens=960 avail_mem=1.19 GB):  26%|██▌       | 13/50 [00:33<00:57,  1.56s/it] 
Capturing prefill shape (num_tokens=960 avail_mem=1.19 GB):  28%|██▊       | 14/50 [00:34<00:57,  1.60s/it]
Capturing prefill shape (num_tokens=896 avail_mem=1.18 GB):  28%|██▊       | 14/50 [00:34<00:57,  1.60s/it]
Capturing prefill shape (num_tokens=896 avail_mem=1.18 GB):  30%|███       | 15/50 [00:35<00:44,  1.28s/it]
Capturing prefill shape (num_tokens=832 avail_mem=1.18 GB):  30%|███       | 15/50 [00:35<00:44,  1.28s/it]
Capturing prefill shape (num_tokens=832 avail_mem=1.18 GB):  32%|███▏      | 16/50 [00:36<00:45,  1.35s/it]
Capturing prefill shape (num_tokens=768 avail_mem=1.17 GB):  32%|███▏      | 16/50 [00:36<00:45,  1.35s/it]
Capturing prefill shape (num_tokens=768 avail_mem=1.17 GB):  34%|███▍      | 17/50 [00:37<00:35,  1.09s/it]
Capturing prefill shape (num_tokens=704 avail_mem=1.17 GB):  34%|███▍      | 17/50 [00:37<00:35,  1.09s/it]
Capturing prefill shape (num_tokens=704 avail_mem=1.17 GB):  36%|███▌      | 18/50 [00:38<00:36,  1.13s/it]
Capturing prefill shape (num_tokens=640 avail_mem=1.16 GB):  36%|███▌      | 18/50 [00:38<00:36,  1.13s/it]
Capturing prefill shape (num_tokens=640 avail_mem=1.16 GB):  38%|███▊      | 19/50 [00:39<00:31,  1.01s/it]
Capturing prefill shape (num_tokens=576 avail_mem=1.15 GB):  38%|███▊      | 19/50 [00:39<00:31,  1.01s/it]
Capturing prefill shape (num_tokens=576 avail_mem=1.15 GB):  40%|████      | 20/50 [00:39<00:23,  1.25it/s]
Capturing prefill shape (num_tokens=512 avail_mem=1.15 GB):  40%|████      | 20/50 [00:39<00:23,  1.25it/s]
Capturing prefill shape (num_tokens=512 avail_mem=1.15 GB):  42%|████▏     | 21/50 [00:40<00:22,  1.30it/s]
Capturing prefill shape (num_tokens=480 avail_mem=1.14 GB):  42%|████▏     | 21/50 [00:40<00:22,  1.30it/s]
Capturing prefill shape (num_tokens=480 avail_mem=1.14 GB):  44%|████▍     | 22/50 [00:40<00:17,  1.64it/s]
Capturing prefill shape (num_tokens=448 avail_mem=1.14 GB):  44%|████▍     | 22/50 [00:40<00:17,  1.64it/s]
Capturing prefill shape (num_tokens=448 avail_mem=1.14 GB):  46%|████▌     | 23/50 [00:41<00:16,  1.61it/s]
Capturing prefill shape (num_tokens=416 avail_mem=1.13 GB):  46%|████▌     | 23/50 [00:41<00:16,  1.61it/s]
Capturing prefill shape (num_tokens=416 avail_mem=1.13 GB):  48%|████▊     | 24/50 [00:41<00:13,  1.95it/s]
Capturing prefill shape (num_tokens=384 avail_mem=1.12 GB):  48%|████▊     | 24/50 [00:41<00:13,  1.95it/s]
Capturing prefill shape (num_tokens=384 avail_mem=1.12 GB):  50%|█████     | 25/50 [00:41<00:13,  1.90it/s]
Capturing prefill shape (num_tokens=352 avail_mem=1.12 GB):  50%|█████     | 25/50 [00:41<00:13,  1.90it/s]
Capturing prefill shape (num_tokens=352 avail_mem=1.12 GB):  52%|█████▏    | 26/50 [00:42<00:11,  2.03it/s]
Capturing prefill shape (num_tokens=320 avail_mem=1.11 GB):  52%|█████▏    | 26/50 [00:42<00:11,  2.03it/s]
Capturing prefill shape (num_tokens=320 avail_mem=1.11 GB):  54%|█████▍    | 27/50 [00:42<00:10,  2.29it/s]
Capturing prefill shape (num_tokens=288 avail_mem=1.11 GB):  54%|█████▍    | 27/50 [00:42<00:10,  2.29it/s]
Capturing prefill shape (num_tokens=288 avail_mem=1.11 GB):  56%|█████▌    | 28/50 [00:43<00:11,  1.91it/s]
Capturing prefill shape (num_tokens=256 avail_mem=1.10 GB):  56%|█████▌    | 28/50 [00:43<00:11,  1.91it/s]
Capturing prefill shape (num_tokens=256 avail_mem=1.10 GB):  58%|█████▊    | 29/50 [00:43<00:09,  2.33it/s]
Capturing prefill shape (num_tokens=240 avail_mem=1.10 GB):  58%|█████▊    | 29/50 [00:43<00:09,  2.33it/s]
Capturing prefill shape (num_tokens=240 avail_mem=1.10 GB):  60%|██████    | 30/50 [00:44<00:08,  2.36it/s]
Capturing prefill shape (num_tokens=224 avail_mem=1.09 GB):  60%|██████    | 30/50 [00:44<00:08,  2.36it/s]
Capturing prefill shape (num_tokens=224 avail_mem=1.09 GB):  62%|██████▏   | 31/50 [00:44<00:07,  2.71it/s]
Capturing prefill shape (num_tokens=208 avail_mem=1.08 GB):  62%|██████▏   | 31/50 [00:44<00:07,  2.71it/s]
Capturing prefill shape (num_tokens=208 avail_mem=1.08 GB):  64%|██████▍   | 32/50 [00:44<00:05,  3.20it/s]
Capturing prefill shape (num_tokens=192 avail_mem=1.08 GB):  64%|██████▍   | 32/50 [00:44<00:05,  3.20it/s]
Capturing prefill shape (num_tokens=192 avail_mem=1.08 GB):  66%|██████▌   | 33/50 [00:44<00:05,  3.21it/s]
Capturing prefill shape (num_tokens=176 avail_mem=1.07 GB):  66%|██████▌   | 33/50 [00:44<00:05,  3.21it/s]
Capturing prefill shape (num_tokens=176 avail_mem=1.07 GB):  68%|██████▊   | 34/50 [00:44<00:04,  3.77it/s]
Capturing prefill shape (num_tokens=160 avail_mem=1.07 GB):  68%|██████▊   | 34/50 [00:44<00:04,  3.77it/s]
Capturing prefill shape (num_tokens=160 avail_mem=1.07 GB):  70%|███████   | 35/50 [00:45<00:04,  3.63it/s]
Capturing prefill shape (num_tokens=144 avail_mem=1.06 GB):  70%|███████   | 35/50 [00:45<00:04,  3.63it/s]
Capturing prefill shape (num_tokens=144 avail_mem=1.06 GB):  72%|███████▏  | 36/50 [00:45<00:03,  4.19it/s]
Capturing prefill shape (num_tokens=128 avail_mem=1.06 GB):  72%|███████▏  | 36/50 [00:45<00:03,  4.19it/s]
Capturing prefill shape (num_tokens=128 avail_mem=1.06 GB):  74%|███████▍  | 37/50 [00:45<00:03,  4.20it/s]
Capturing prefill shape (num_tokens=112 avail_mem=1.04 GB):  74%|███████▍  | 37/50 [00:45<00:03,  4.20it/s]
Capturing prefill shape (num_tokens=112 avail_mem=1.04 GB):  76%|███████▌  | 38/50 [00:45<00:02,  4.76it/s]
Capturing prefill shape (num_tokens=96 avail_mem=1.03 GB):  76%|███████▌  | 38/50 [00:45<00:02,  4.76it/s] 
Capturing prefill shape (num_tokens=96 avail_mem=1.03 GB):  78%|███████▊  | 39/50 [00:45<00:02,  5.36it/s]
Capturing prefill shape (num_tokens=80 avail_mem=1.03 GB):  78%|███████▊  | 39/50 [00:45<00:02,  5.36it/s]
Capturing prefill shape (num_tokens=80 avail_mem=1.03 GB):  80%|████████  | 40/50 [00:46<00:01,  5.57it/s]
Capturing prefill shape (num_tokens=64 avail_mem=1.02 GB):  80%|████████  | 40/50 [00:46<00:01,  5.57it/s]
Capturing prefill shape (num_tokens=64 avail_mem=1.02 GB):  82%|████████▏ | 41/50 [00:46<00:01,  6.10it/s]
Capturing prefill shape (num_tokens=48 avail_mem=1.02 GB):  82%|████████▏ | 41/50 [00:46<00:01,  6.10it/s]
Capturing prefill shape (num_tokens=48 avail_mem=1.02 GB):  84%|████████▍ | 42/50 [00:46<00:01,  6.06it/s]
Capturing prefill shape (num_tokens=32 avail_mem=1.01 GB):  84%|████████▍ | 42/50 [00:46<00:01,  6.06it/s]
Capturing prefill shape (num_tokens=32 avail_mem=1.01 GB):  86%|████████▌ | 43/50 [00:46<00:01,  6.54it/s]
Capturing prefill shape (num_tokens=28 avail_mem=1.00 GB):  86%|████████▌ | 43/50 [00:46<00:01,  6.54it/s]
Capturing prefill shape (num_tokens=28 avail_mem=1.00 GB):  88%|████████▊ | 44/50 [00:46<00:00,  6.29it/s]
Capturing prefill shape (num_tokens=24 avail_mem=1.00 GB):  88%|████████▊ | 44/50 [00:46<00:00,  6.29it/s]
Capturing prefill shape (num_tokens=24 avail_mem=1.00 GB):  90%|█████████ | 45/50 [00:46<00:00,  6.53it/s]
Capturing prefill shape (num_tokens=20 avail_mem=0.99 GB):  90%|█████████ | 45/50 [00:46<00:00,  6.53it/s]
Capturing prefill shape (num_tokens=20 avail_mem=0.99 GB):  92%|█████████▏| 46/50 [00:46<00:00,  6.92it/s]
Capturing prefill shape (num_tokens=16 avail_mem=0.99 GB):  92%|█████████▏| 46/50 [00:46<00:00,  6.92it/s]
Capturing prefill shape (num_tokens=16 avail_mem=0.99 GB):  94%|█████████▍| 47/50 [00:47<00:00,  6.08it/s]
Capturing prefill shape (num_tokens=12 avail_mem=0.98 GB):  94%|█████████▍| 47/50 [00:47<00:00,  6.08it/s]
Capturing prefill shape (num_tokens=12 avail_mem=0.98 GB):  96%|█████████▌| 48/50 [00:47<00:00,  6.64it/s]
Capturing prefill shape (num_tokens=8 avail_mem=0.98 GB):  96%|█████████▌| 48/50 [00:47<00:00,  6.64it/s] 
Capturing prefill shape (num_tokens=8 avail_mem=0.98 GB):  98%|█████████▊| 49/50 [00:47<00:00,  6.68it/s]
Capturing prefill shape (num_tokens=4 avail_mem=0.97 GB):  98%|█████████▊| 49/50 [00:47<00:00,  6.68it/s]
Capturing prefill shape (num_tokens=4 avail_mem=0.97 GB): 100%|██████████| 50/50 [00:47<00:00,  7.10it/s]
Capturing prefill shape (num_tokens=4 avail_mem=0.97 GB): 100%|██████████| 50/50 [00:47<00:00,  1.05it/s]
[2026-10-17 14:43:33] Capture target prefill CUDA graph end. elapsed=48.33 s, mem usage=0.83 GB, avail mem=1.45 GB.
