VLLM / A3GLM-5.1 · A3
每种布局独立记录镜像与脚本。先选拓扑，再核对具体的部署条件。
镜像TODO
GLM-5.1-w8a8 (Eco-Tech/GLM-5.1-w8a8), single A3 node (16 NPU), DP=1 x TP=16. The GLM5.html page covers GLM-5 and GLM-5.1 with identical commands (model weights differ only); extracted the A3 w8a8 single-node block. served-model-name glm-5 (upstream; the name clients call). Speculative method deepseek_mtp is GLM-5/5.1 own MTP per upstream. Deviations: /root/model (not modelscope cache vllm-ascend/GLM5-w8a8), --port 8080 (upstream 8077). env (HCCL_*, VLLM_ASCEND_BALANCE_SCHEDULING/MLAPO/FLASHCOMM1) and serve flags kept verbatim.
脚本与说明
来源记录日期 2026-06-28 · 此页面不重新认定硬件验证结果
A3 / LAYOUT
1x2p1x2d
当前版本待验证镜像TODO
GLM-5.1-w8a8 (Eco-Tech/GLM-5.1-w8a8) PD on 4x A3 (16 NPU/node): 1 prefill engine spanning 2 nodes (DP=2 x TP=16, dp-local=1, 1 instance/node, kv_producer 30000) + 1 decode engine spanning 2 nodes (DP=8 x TP=4, dp-local=4, 4 instances/node, kv_consumer 30100). This is the half-decode sibling of 1x2p1x4d: prefill side is IDENTICAL (DP=2 x TP=16), decode shrunk from 4 nodes (DP=16) to 2 nodes (DP=8); per-decode-instance sizing (TP=4, 4 NPU, max-num-seqs 8) is unchanged. External online DP (launch_online_dp.py + run_dp_template_{prefill,decode}.sh with --data-parallel-rank per instance). MooncakeConnectorV1, kv_connector_extra_config prefill{dp 2,tp 16}/decode{dp 8,tp 4} on BOTH roles, use_ascend_direct. Proxy load_balance_proxy_server_example.py (non-layerwise, vendored) fans 2 prefill + 8 decode endpoints. tool-call-parser glm47, reasoning-parser glm45, speculative deepseek_mtp. The GLM5.html page covers GLM-5/GLM-5.1 with identical commands; extracted the w8a8 PD block. Deviations from upstream: /root/model; NIC/IP + master addrs resolved at runtime via the spine; rpc-ports 10521 (prefill) / 10523 (decode) and vllm-start-ports 9081 / 9900 kept; added LD_LIBRARY_PATH=/usr/local/lib for Mooncake ascend_transport.so (ModelArts sh launch); DROPPED upstream --profiler-config (a debug artifact, not a serving flag). kv_ports per upstream. Note: prefill --max-model-len 131072 vs decode 200000 is asymmetric BY DESIGN per upstream 5.3 -- not an introduced inconsistency; do not "fix" to match. The smaller decode is NOT separately documented upstream (upstream shows the 4-node decode); treat as a derived starting point -- the decode DP=8 fan-out and the dp_size=8 in kv_connector_extra_config are the only changes vs the validated-shape 1x2p1x4d.
脚本与说明
来源记录日期 2026-06-30 · 此页面不重新认定硬件验证结果
A3 / LAYOUT
1x2p1x4d
当前版本待验证镜像TODO
GLM-5.1-w8a8 (Eco-Tech/GLM-5.1-w8a8) PD on 6x A3 (16 NPU/node): 1 prefill engine spanning 2 nodes (DP=2 x TP=16, dp-local=1, 1 instance/node, kv_producer 30000) + 1 decode engine spanning 4 nodes (DP=16 x TP=4, dp-local=4, 4 instances/node, kv_consumer 30100). External online DP (launch_online_dp.py + run_dp_template_{prefill,decode}.sh with --data-parallel-rank per instance). MooncakeConnectorV1, kv_connector_extra_config prefill{dp 2,tp 16}/decode{dp 16,tp 4}, use_ascend_direct. Proxy load_balance_proxy_server_example.py (non-layerwise, vendored) fans 2 prefill + 16 decode endpoints. tool-call-parser glm47, reasoning-parser glm45, speculative deepseek_mtp. The GLM5.html page covers GLM-5/GLM-5.1 with identical commands; extracted the w8a8 PD block. Deviations from upstream: /root/model; NIC/IP + master addrs resolved at runtime via the spine; rpc-ports 10521 (prefill) / 10523 (decode) and vllm-start-ports 9081 / 9900 kept; added LD_LIBRARY_PATH=/usr/local/lib for Mooncake ascend_transport.so (ModelArts sh launch); DROPPED upstream --profiler-config (forces torch profiling to ./vllm_profile on every instance -- a debug artifact, not a serving flag). kv_ports/dp-tp sizes per upstream. Note: prefill --max-model-len 131072 vs decode 200000 is asymmetric BY DESIGN per upstream 5.3 (prefill run_p.sh=131072, decode run_d*.sh=200000) -- not an introduced inconsistency; do not "fix" to match.
脚本与说明
来源记录日期 2026-06-28 · 此页面不重新认定硬件验证结果
镜像TODO
GLM-5.1-w8a8 (Eco-Tech/GLM-5.1-w8a8), 2x A3 (16 NPU/node) single mixed engine, DP=2 x TP=16 (dp-local=1, one DP rank per node). rank0=leader (API :8080), rank1=headless. The GLM5.html page covers GLM-5/GLM-5.1 with identical commands; extracted the A3 w8a8 multi-node block. served-model-name glm-5, speculative deepseek_mtp (GLM own MTP). headless node keeps --data-parallel-start-rank (no per-node API server). Deviations: /root/model (not vllm-ascend/GLM5-w8a8), --port 8080 (upstream 8077); NIC/IP resolved at runtime via the spine (upstream had placeholder nic_name/local_ip/node0_ip).
脚本与说明
来源记录日期 2026-06-28 · 此页面不重新认定硬件验证结果