VLLM / A3Qwen3.5-397B-A17B · A3
每种布局独立记录镜像与脚本。先选拓扑，再核对具体的部署条件。
镜像TODO
Qwen3.5-397B-A17B-w8a8-mtp, single A3 node (16 NPU), DP=1 x TP=16. served-model-name qwen3.5 (upstream). Speculative method qwen3_5_mtp is Qwen3.5 own MTP per upstream. Deviations from upstream 5.1: /root/model (not modelscope cache Eco-Tech/Qwen3.5-397B-A17B-w8a8-mtp), --port 8080 (upstream 8000), dropped VLLM_USE_MODELSCOPE (model pre-mounted). All other env (HCCL_*, jemalloc LD_PRELOAD, cpufreq/sysctl tuning) and serve flags kept verbatim.
脚本与说明
来源记录日期 2026-06-28 · 此页面不重新认定硬件验证结果
镜像TODO
Qwen3.5-397B-A17B-w8a8-mtp PD: 1 single-node prefill (DP=8 x TP=2, kv_producer, kv_port 23010) + 1 decode spanning 2 A3 nodes (DP=16 x TP=2, kv_consumer, kv_port 36010). MooncakeLayerwiseConnector (NOT MooncakeConnectorV1) with kv_connector_extra_config prefill{dp 8,tp 2}/decode{dp 16,tp 2}; proxy is load_balance_proxy_layerwise_server_example.py (layerwise variant, vendored from vllm-ascend main; falls back to the image copy). rank0=prefill, rank1=decode master (API), rank2=decode headless. served-model-name qwen3.5, speculative qwen3_5_mtp. Deviations from upstream 5.4: /root/model; engine API ports unified to 7100 (upstream 30060/30050) so proxy wiring derives from AISHIPBOX_ADDRS; NIC/IP resolved at runtime via the spine; added LD_LIBRARY_PATH=/usr/local/lib for Mooncake ascend_transport.so (ModelArts sh launch). kv_port values, connector type, dp/tp sizes, env block kept per upstream. KNOWN RISK: the decode master (run_decode_node0.sh) runs an API server (--api-server-count 1) yet uses --data-parallel-start-rank 0, which CLAUDE.md/[[dp-rank-vs-start-rank]] warns can cause a ~5-min front-end handshake timeout for per-node-API-server multi-node DP. Kept as-is because it is upstream-faithful (Qwen 5.4 run_d0.sh uses start-rank 0); the verified deepseek 2p1x2d decode master passes NO start-rank flag instead. If decode never becomes ready on bring-up, try dropping --data-parallel-start-rank 0 on the master (keep it only on the headless worker run_decode_node1.sh).
脚本与说明
来源记录日期 2026-06-28 · 此页面不重新认定硬件验证结果