{ "schema_version": 1, "snapshot_date": "2026-10-23", "sources": [ { "repository": "https://github.com/xuopoj/ascend-docker-factory", "commit": "c927e89fcac2d6622ab1b1463934c502f652e61a", "local_changes": true }, { "repository": "https://github.com/xuopoj/ma-vllm-samples", "commit": "403c1f18cba35f9c81c75359632fc4954cecf509", "local_changes": false } ], "images": [ { "id": "cann-8.3rc1-310p", "name": "cann", "tag": "quay.io/service-delivery-hub/cann:8.3rc1-310p", "chip": "310p", "cann_version": "8.3.RC1", "purpose": null, "build_args": { "CANN_VERSION": "8.3.RC1", "CHIP_TYPE": "310p", "DRIVER_VERSION": "24.1.1", "INSTALL_COMPONENTS": "toolkit,kernels" }, "pushed_at": "2026-03-23T02:41:16.813159+00:00", "offline_url": "https://modelscope.cn/datasets/xuopoj/service-delivery-hub/resolve/master/cann/cann-8.3rc1-310p.tar.gz", "verification": "build-record-only", "verification_note": "来源包含构建配置和推送时间，未附真机执行证明；不标记为硬件已验证。", "source_path": "catalog/cann-8.3rc1-310p.json", "source_url": "https://github.com/xuopoj/ascend-docker-factory/blob/c927e89fcac2d6622ab1b1463934c502f652e61a/catalog/cann-8.3rc1-310p.json", "matches_source_commit": true, "source_file_sha256": "9430bdc22d31e05e5a425853363ee309cb8e22245f1b2c7c2180e8aab2d0658c" }, { "id": "cann-8.3rc1-910b", "name": "cann", "tag": "quay.io/service-delivery-hub/cann:8.3rc1-910b", "chip": "910b", "cann_version": "8.3.RC1", "purpose": null, "build_args": { "CANN_VERSION": "8.3.RC1", "CHIP_TYPE": "910b", "DRIVER_VERSION": "24.1.1", "INSTALL_COMPONENTS": "toolkit,kernels,nnal" }, "pushed_at": "2026-03-23T01:46:30.460987+00:00", "offline_url": "https://modelscope.cn/datasets/xuopoj/service-delivery-hub/resolve/master/cann/cann-8.3rc1-910b.tar.gz", "verification": "build-record-only", "verification_note": "来源包含构建配置和推送时间，未附真机执行证明；不标记为硬件已验证。", "source_path": "catalog/cann-8.3rc1-910b.json", "source_url": "https://github.com/xuopoj/ascend-docker-factory/blob/c927e89fcac2d6622ab1b1463934c502f652e61a/catalog/cann-8.3rc1-910b.json", "matches_source_commit": true, "source_file_sha256": "ddc4f4e7682756386a9d9cc3475b945f0f100cb4b50f814908ff235b3a88c899" }, { "id": "python-3.10", "name": "python", "tag": "quay.io/service-delivery-hub/python:3.10", "chip": null, "cann_version": null, "purpose": null, "build_args": { "BASE_IMAGE": "ubuntu:22.04", "PIP_INDEX_URL": "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/simple", "INCLUDE_DEBUG_TOOLS": "true", "PYTHON_VERSION": "3.10" }, "pushed_at": "2026-03-23T02:13:15.402281+00:00", "offline_url": "https://modelscope.cn/datasets/xuopoj/service-delivery-hub/resolve/master/python/python-3.10.tar.gz", "verification": "build-record-only", "verification_note": "来源包含构建配置和推送时间，未附真机执行证明；不标记为硬件已验证。", "source_path": "catalog/python-3.10.json", "source_url": "https://github.com/xuopoj/ascend-docker-factory/blob/c927e89fcac2d6622ab1b1463934c502f652e61a/catalog/python-3.10.json", "matches_source_commit": true, "source_file_sha256": "c39506ae53c4d5797e4931018434288e5f463156ef4fc13746c1fc67f56d0d4c" }, { "id": "python-3.11", "name": "python", "tag": "quay.io/service-delivery-hub/python:3.11", "chip": null, "cann_version": null, "purpose": null, "build_args": { "BASE_IMAGE": "ubuntu:22.04", "PIP_INDEX_URL": "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/simple", "INCLUDE_DEBUG_TOOLS": "true", "PYTHON_VERSION": "3.11" }, "pushed_at": "2026-03-22T15:35:45.888520+00:00", "offline_url": "https://modelscope.cn/datasets/xuopoj/service-delivery-hub/resolve/master/python/python-3.11.tar.gz", "verification": "build-record-only", "verification_note": "来源包含构建配置和推送时间，未附真机执行证明；不标记为硬件已验证。", "source_path": "catalog/python-3.11.json", "source_url": "https://github.com/xuopoj/ascend-docker-factory/blob/c927e89fcac2d6622ab1b1463934c502f652e61a/catalog/python-3.11.json", "matches_source_commit": true, "source_file_sha256": "98cd6c771fba2de3845f15dd92fa4dee5469bbc183a6c286a15d4354371db63c" }, { "id": "vllm-ascend-v0.13.0", "name": "vllm", "tag": "quay.io/service-delivery-hub/vllm-ascend:v0.13.0", "chip": null, "cann_version": null, "purpose": "inference", "build_args": { "INCLUDE_DEBUG_TOOLS": "true", "BASE_IMAGE": "v0.13.0" }, "pushed_at": "2026-03-22T12:07:18.941554+00:00", "offline_url": "https://modelscope.cn/datasets/xuopoj/service-delivery-hub/resolve/master/vllm-ascend/vllm-ascend-v0.13.0.tar.gz", "verification": "build-record-only", "verification_note": "来源包含构建配置和推送时间，未附真机执行证明；不标记为硬件已验证。", "source_path": "catalog/vllm-ascend-v0.13.0.json", "source_url": "https://github.com/xuopoj/ascend-docker-factory/blob/c927e89fcac2d6622ab1b1463934c502f652e61a/catalog/vllm-ascend-v0.13.0.json", "matches_source_commit": true, "source_file_sha256": "eb7e4548550be777a99f1fdfe282ebfc32047b32973dac50b91fc45ed7464c40" }, { "id": "vllm-ascend-v0.14.0rc1", "name": "vllm", "tag": "quay.io/service-delivery-hub/vllm-ascend:v0.14.0rc1", "chip": null, "cann_version": null, "purpose": "inference", "build_args": { "INCLUDE_DEBUG_TOOLS": "true", "BASE_IMAGE": "v0.14.0rc1" }, "pushed_at": "2026-03-22T14:09:23.301732+00:00", "offline_url": "https://modelscope.cn/datasets/xuopoj/service-delivery-hub/resolve/master/vllm-ascend/vllm-ascend-v0.14.0rc1.tar.gz", "verification": "build-record-only", "verification_note": "来源包含构建配置和推送时间，未附真机执行证明；不标记为硬件已验证。", "source_path": "catalog/vllm-ascend-v0.14.0rc1.json", "source_url": "https://github.com/xuopoj/ascend-docker-factory/blob/c927e89fcac2d6622ab1b1463934c502f652e61a/catalog/vllm-ascend-v0.14.0rc1.json", "matches_source_commit": true, "source_file_sha256": "71ca91d379583a751bc41d976c71eb1b19f2e8e43aed72c6835e618b424833a0" }, { "id": "vllm-ascend-v0.15.0rc1", "name": "vllm", "tag": "quay.io/service-delivery-hub/vllm-ascend:v0.15.0rc1", "chip": null, "cann_version": null, "purpose": "inference", "build_args": { "INCLUDE_DEBUG_TOOLS": "true", "BASE_IMAGE": "v0.15.0rc1" }, "pushed_at": "2026-03-22T11:28:19.759906+00:00", "offline_url": "https://modelscope.cn/datasets/xuopoj/service-delivery-hub/resolve/master/vllm-ascend/vllm-ascend-v0.15.0rc1.tar.gz", "verification": "build-record-only", "verification_note": "来源包含构建配置和推送时间，未附真机执行证明；不标记为硬件已验证。", "source_path": "catalog/vllm-ascend-v0.15.0rc1.json", "source_url": "https://github.com/xuopoj/ascend-docker-factory/blob/c927e89fcac2d6622ab1b1463934c502f652e61a/catalog/vllm-ascend-v0.15.0rc1.json", "matches_source_commit": true, "source_file_sha256": "90d6343f497c6c2295e3e9e0a25a3a0c81f4dc312024f32953e06567e0e04e88" }, { "id": "vllm-ascend-v0.17.0rc1", "name": "vllm", "tag": "quay.io/service-delivery-hub/vllm-ascend:v0.17.0rc1", "chip": null, "cann_version": null, "purpose": "inference", "build_args": { "INCLUDE_DEBUG_TOOLS": "true", "BASE_IMAGE": "v0.17.0rc1" }, "pushed_at": "2026-03-22T03:58:48.211063+00:00", "offline_url": "https://modelscope.cn/datasets/xuopoj/service-delivery-hub/resolve/master/vllm-ascend/vllm-ascend-v0.17.0rc1.tar.gz", "verification": "build-record-only", "verification_note": "来源包含构建配置和推送时间，未附真机执行证明；不标记为硬件已验证。", "source_path": "catalog/vllm-ascend-v0.17.0rc1.json", "source_url": "https://github.com/xuopoj/ascend-docker-factory/blob/c927e89fcac2d6622ab1b1463934c502f652e61a/catalog/vllm-ascend-v0.17.0rc1.json", "matches_source_commit": true, "source_file_sha256": "24309a9a71cb23bb0c809d3c704d57269ddcf66c8f17b29f18250ce3128cae40" } ], "models": [ { "id": "deepseekv4-flash-a2", "title": "DeepSeek-V4-Flash · A2", "platform": "A2", "layouts": [ { "layout": "1x2p2d", "image": "quay.io/ascend/vllm-ascend:v0.21.0rc1", "verified": false, "historical_use": false, "derived": "2026-06-27", "upstream": "https://docs.vllm.ai/projects/ascend/en/latest/tutorials/models/DeepSeek-V4-Flash.html", "note": "4x A2: 1 prefill (DP=16) spanning nodes 0-1 + 2 standalone decode (DP=8), MooncakeHybridConnector, kv_port 30000/30200/30300 (engine_id 0/2/3). v0.21.0rc1 refresh: migrated speculative method deepseek_mtp -> mtp (enforce_eager: true; decode keeps num_speculative_tokens 2) and dropped dead env toggles (USE_MULTI_BLOCK_POOL, VLLM_ASCEND_ENABLE_FUSED_MC2, USE_MULTI_GROUPS_KV_CACHE). Cross-node prefill topology, A2 memory tuning, and --chat-template preserved.", "files": [ "check_hccn.sh", "load_balance_proxy_server_example.py", "meta.yaml", "run.sh", "run_decode_node0.sh", "run_decode_node1.sh", "run_prefill_node0.sh", "run_prefill_node1.sh", "run_proxy.sh", "setup_rank_env.sh", "smoke_test.py" ], "source_path": "models/deepseekv4-flash-a2/1x2p2d", "source_url": "https://github.com/xuopoj/ma-vllm-samples/tree/403c1f18cba35f9c81c75359632fc4954cecf509/models/deepseekv4-flash-a2/1x2p2d" }, { "layout": "2nodes", "image": "quay.io/ascend/vllm-ascend:v0.21.0rc1", "verified": false, "historical_use": false, "derived": "2026-06-27", "upstream": "https://docs.vllm.ai/projects/ascend/en/latest/tutorials/models/DeepSeek-V4-Flash.html", "note": "2x A2 (8 NPU/node) single mixed engine, DP=16 x TP=1 x EP=16 across both nodes. v0.21.0rc1 refresh: migrated speculative method deepseek_mtp -> mtp (enforce_eager: true) and dropped dead env toggles (USE_MULTI_BLOCK_POOL, VLLM_ASCEND_ENABLE_FUSED_MC2, USE_MULTI_GROUPS_KV_CACHE). A2 memory tuning (max-model-len 131072, gpu-mem 0.90) and --chat-template preserved.", "files": [ "check_hccn.sh", "meta.yaml", "run.sh", "run_node0.sh", "run_node1.sh", "setup_rank_env.sh" ], "source_path": "models/deepseekv4-flash-a2/2nodes", "source_url": "https://github.com/xuopoj/ma-vllm-samples/tree/403c1f18cba35f9c81c75359632fc4954cecf509/models/deepseekv4-flash-a2/2nodes" }, { "layout": "2p1x2d", "image": "quay.io/ascend/vllm-ascend:v0.21.0rc1", "verified": false, "historical_use": false, "derived": "2026-06-27", "upstream": "./README.md", "note": ">", "files": [ "README.md", "check_hccn.sh", "meta.yaml", "run.sh", "run_decode_node0.sh", "run_decode_node1.sh", "run_prefill_node0.sh", "run_prefill_node1.sh", "run_proxy.sh", "setup_rank_env.sh" ], "source_path": "models/deepseekv4-flash-a2/2p1x2d", "source_url": "https://github.com/xuopoj/ma-vllm-samples/tree/403c1f18cba35f9c81c75359632fc4954cecf509/models/deepseekv4-flash-a2/2p1x2d" }, { "layout": "2p2d", "image": "quay.io/ascend/vllm-ascend:v0.21.0rc1", "verified": false, "historical_use": false, "derived": "2026-06-27", "upstream": "https://docs.vllm.ai/projects/ascend/en/latest/tutorials/models/DeepSeek-V4-Flash.html", "note": "4x A2: 2 standalone prefill (DP=8) + 2 standalone decode (DP=8), MooncakeHybridConnector, kv_port 30000/30100/30200/30300 (engine_id 0..3). v0.21.0rc1 refresh: migrated speculative method deepseek_mtp -> mtp (enforce_eager: true; decode keeps num_speculative_tokens 2) and dropped dead env toggles (USE_MULTI_BLOCK_POOL, VLLM_ASCEND_ENABLE_FUSED_MC2, USE_MULTI_GROUPS_KV_CACHE). Repo-specific symmetric topology, A2 memory tuning, and --chat-template preserved.", "files": [ "check_hccn.sh", "load_balance_proxy_server_example.py", "meta.yaml", "run.sh", "run_decode_node0.sh", "run_decode_node1.sh", "run_prefill_node0.sh", "run_prefill_node1.sh", "run_proxy.sh", "setup_rank_env.sh" ], "source_path": "models/deepseekv4-flash-a2/2p2d", "source_url": "https://github.com/xuopoj/ma-vllm-samples/tree/403c1f18cba35f9c81c75359632fc4954cecf509/models/deepseekv4-flash-a2/2p2d" } ] }, { "id": "deepseekv4-flash-a3", "title": "DeepSeek-V4-Flash · A3", "platform": "A3", "layouts": [ { "layout": "1node", "image": "quay.io/ascend/vllm-ascend:v0.21.0rc1-a3", "verified": false, "historical_use": true, "derived": "2026-06-27", "upstream": "https://docs.vllm.ai/projects/ascend/en/latest/tutorials/models/DeepSeek-V4-Flash.html", "note": "Single-node A3, DP=4 x TP=4. Already aligned with the v0.21.0rc1 reference (mtp/enforce_eager); this pass only refreshes provenance. Was hardware-verified at an earlier image; flipped to false until re-run on v0.21.0rc1.", "files": [ "check_hccn.sh", "meta.yaml", "run.sh" ], "source_path": "models/deepseekv4-flash-a3/1node", "source_url": "https://github.com/xuopoj/ma-vllm-samples/tree/403c1f18cba35f9c81c75359632fc4954cecf509/models/deepseekv4-flash-a3/1node", "historical_evidence": "https://github.com/xuopoj/ma-vllm-samples/blob/403c1f18cba35f9c81c75359632fc4954cecf509/README.md" }, { "layout": "1p1d", "image": "quay.io/ascend/vllm-ascend:v0.21.0rc1-a3", "verified": false, "historical_use": true, "derived": "2026-06-27", "upstream": "https://docs.vllm.ai/projects/ascend/en/latest/tutorials/models/DeepSeek-V4-Flash.html", "note": "A3 1P1D PD separation with external online DP. Already aligned with the v0.21.0rc1 reference (mtp/enforce_eager). kv_port deviates to 36000/36100 (upstream 30000/30100): the official kv_port table reserves [20000,36000) for AscendDirectTransport on 16-NPU nodes. Was hardware-verified at an earlier image; flipped to false until re-run on v0.21.0rc1.", "files": [ "README.md", "check_hccn.sh", "launch_online_dp.py", "load_balance_proxy_server_example.py", "meta.yaml", "metrics_aggregator.py", "run.sh", "run_decode_node0.sh", "run_dp_template_decode.sh", "run_dp_template_prefill.sh", "run_prefill_node0.sh", "run_proxy.sh", "setup_rank_env.sh", "smoke_test.py" ], "source_path": "models/deepseekv4-flash-a3/1p1d", "source_url": "https://github.com/xuopoj/ma-vllm-samples/tree/403c1f18cba35f9c81c75359632fc4954cecf509/models/deepseekv4-flash-a3/1p1d", "historical_evidence": "https://github.com/xuopoj/ma-vllm-samples/blob/403c1f18cba35f9c81c75359632fc4954cecf509/README.md" }, { "layout": "1p3d", "image": "quay.io/ascend/vllm-ascend:v0.21.0rc1-a3", "verified": false, "historical_use": false, "derived": "2026-06-30", "upstream": "https://docs.vllm.ai/projects/ascend/en/latest/tutorials/models/DeepSeek-V4-Flash.html", "note": "DeepSeek-V4-Flash-w8a8-mtp PD on 4x A3 (16 NPU/node), decode-heavy: 1 single-node prefill engine (DP=4 x TP=4, 4 instances 7100..7103, kv_producer, engine_id 0) + 3 INDEPENDENT single-node decode engines (each DP=16 x TP=1, 16 instances 7100..7115, kv_consumer, engine_id 1/3/5). External online DP (launch_online_dp.py + parameterized run_dp_template_{prefill,decode}.sh; ENGINE_ID/KV_PORT exported per node by run.sh). MooncakeHybridConnector, kv_connector_extra_config prefill{dp 4,tp 4}/decode{dp 16,tp 1} on every engine. Proxy load_balance_proxy_server_example.py (non-layerwise, vendored) fans 4 prefill + 48 decode endpoints. DERIVED from the verified 1p1d layout by running multiple independent same-role engines; upstream documents only 1P1D, so the multi-engine fan-out is NOT separately validated. Per-engine sizing/flags identical to 1p1d. engine_id globally unique (prefill even 0, decode odd 1/3/5); kv_port stays 36000/36100 because engines are on distinct nodes. kv_port 36000/36100 deviate from upstream 30000/30100 (official table reserves [20000,36000) for AscendDirectTransport on 16-NPU nodes). Use this decode-heavy ratio when decode is the bottleneck (short prompts, long output length) -- the usual PD imbalance for chat workloads. Deviations from upstream: /root/model; NIC/IP resolved at runtime via the spine; LD_LIBRARY_PATH=/usr/local/lib for Mooncake. Treat as a starting point.", "files": [ "check_hccn.sh", "launch_online_dp.py", "load_balance_proxy_server_example.py", "meta.yaml", "run.sh", "run_decode.sh", "run_dp_template_decode.sh", "run_dp_template_prefill.sh", "run_prefill.sh", "run_proxy.sh", "setup_rank_env.sh", "smoke_test.py" ], "source_path": "models/deepseekv4-flash-a3/1p3d", "source_url": "https://github.com/xuopoj/ma-vllm-samples/tree/403c1f18cba35f9c81c75359632fc4954cecf509/models/deepseekv4-flash-a3/1p3d" }, { "layout": "2nodes", "image": "quay.io/ascend/vllm-ascend:v0.21.0rc1-a3", "verified": false, "historical_use": true, "derived": "2026-06-27", "upstream": "https://docs.vllm.ai/projects/ascend/en/latest/tutorials/models/DeepSeek-V4-Flash.html", "note": "2x A3 mixed engine, DP=8 x TP=4 (a3/1node doubled across 2 nodes). v0.21.0rc1 refresh: migrated speculative method deepseek_mtp -> mtp (enforce_eager: true) and dropped dead env toggles (ACL_OP_INIT_MODE, ASCEND_A3_ENABLE, USE_MULTI_BLOCK_POOL, VLLM_ASCEND_ENABLE_FUSED_MC2, USE_MULTI_GROUPS_KV_CACHE). Was hardware-verified at an earlier image; flipped to false until re-run on v0.21.0rc1.", "files": [ "check_hccn.sh", "meta.yaml", "run.sh", "run_node0.sh", "run_node1.sh", "setup_rank_env.sh" ], "source_path": "models/deepseekv4-flash-a3/2nodes", "source_url": "https://github.com/xuopoj/ma-vllm-samples/tree/403c1f18cba35f9c81c75359632fc4954cecf509/models/deepseekv4-flash-a3/2nodes", "historical_evidence": "https://github.com/xuopoj/ma-vllm-samples/blob/403c1f18cba35f9c81c75359632fc4954cecf509/README.md" }, { "layout": "2p1x2d", "image": "quay.io/ascend/vllm-ascend:v0.21.0rc1-a3", "verified": false, "historical_use": false, "derived": "2026-06-27", "upstream": "https://docs.vllm.ai/projects/ascend/en/latest/tutorials/models/DeepSeek-V4-Flash.html", "note": "2 single-node prefill (DP=16) + 1 decode spanning 2 A3 nodes (DP=32), MooncakeConnectorV1, kv_port 30000/30100/30200. v0.21.0rc1 refresh: migrated speculative method deepseek_mtp -> mtp (enforce_eager: true; decode keeps num_speculative_tokens 2) and dropped dead env toggles (USE_MULTI_BLOCK_POOL, VLLM_ASCEND_ENABLE_FUSED_MC2). Topology/ports/engine_ids preserved.", "files": [ "check_hccn.sh", "meta.yaml", "run.sh", "run_decode_node0.sh", "run_decode_node1.sh", "run_prefill_node0.sh", "run_prefill_node1.sh", "run_proxy.sh", "setup_rank_env.sh" ], "source_path": "models/deepseekv4-flash-a3/2p1x2d", "source_url": "https://github.com/xuopoj/ma-vllm-samples/tree/403c1f18cba35f9c81c75359632fc4954cecf509/models/deepseekv4-flash-a3/2p1x2d" }, { "layout": "2p2d", "image": "quay.io/ascend/vllm-ascend:v0.21.0rc1-a3", "verified": false, "historical_use": false, "derived": "2026-06-30", "upstream": "https://docs.vllm.ai/projects/ascend/en/latest/tutorials/models/DeepSeek-V4-Flash.html", "note": "DeepSeek-V4-Flash-w8a8-mtp PD on 4x A3 (16 NPU/node): 2 INDEPENDENT single-node prefill engines (each DP=4 x TP=4, 4 instances 7100..7103, kv_producer, engine_id 0/2) + 2 INDEPENDENT single-node decode engines (each DP=16 x TP=1, 16 instances 7100..7115, kv_consumer, engine_id 1/3). External online DP (launch_online_dp.py + parameterized run_dp_template_{prefill,decode}.sh; ENGINE_ID/KV_PORT exported per node by run.sh). MooncakeHybridConnector, kv_connector_extra_config prefill{dp 4,tp 4}/decode{dp 16,tp 1} on every engine. Proxy load_balance_proxy_server_example.py (non-layerwise, vendored) fans 8 prefill + 32 decode endpoints. DERIVED from the verified 1p1d layout by running multiple independent same-role engines; upstream documents only 1P1D, so the multi-engine fan-out is NOT separately validated. Per-engine sizing/flags are identical to 1p1d. engine_id is globally unique (prefill even, decode odd); kv_port stays 36000/36100 because engines are on distinct nodes (only engine_id must differ). kv_port 36000/36100 deviate from upstream 30000/30100: the official kv_port table reserves [20000,36000) for AscendDirectTransport on 16-NPU nodes. Deviations from upstream: /root/model; NIC/IP resolved at runtime via the spine; LD_LIBRARY_PATH=/usr/local/lib for Mooncake ascend_transport.so (ModelArts sh launch). Treat as a starting point: a multi-P/multi-D fan-out can still hit a KV-transport clash or proxy mis-route that 1p1d would not.", "files": [ "check_hccn.sh", "launch_online_dp.py", "load_balance_proxy_server_example.py", "meta.yaml", "run.sh", "run_decode.sh", "run_dp_template_decode.sh", "run_dp_template_prefill.sh", "run_prefill.sh", "run_proxy.sh", "setup_rank_env.sh", "smoke_test.py" ], "source_path": "models/deepseekv4-flash-a3/2p2d", "source_url": "https://github.com/xuopoj/ma-vllm-samples/tree/403c1f18cba35f9c81c75359632fc4954cecf509/models/deepseekv4-flash-a3/2p2d" }, { "layout": "3p1d", "image": "quay.io/ascend/vllm-ascend:v0.21.0rc1-a3", "verified": false, "historical_use": false, "derived": "2026-06-30", "upstream": "https://docs.vllm.ai/projects/ascend/en/latest/tutorials/models/DeepSeek-V4-Flash.html", "note": "DeepSeek-V4-Flash-w8a8-mtp PD on 4x A3 (16 NPU/node), prefill-heavy: 3 INDEPENDENT single-node prefill engines (each DP=4 x TP=4, 4 instances 7100..7103, kv_producer, engine_id 0/2/4) + 1 single-node decode engine (DP=16 x TP=1, 16 instances 7100..7115, kv_consumer, engine_id 1). External online DP (launch_online_dp.py + parameterized run_dp_template_{prefill,decode}.sh; ENGINE_ID/KV_PORT exported per node by run.sh). MooncakeHybridConnector, kv_connector_extra_config prefill{dp 4,tp 4}/decode{dp 16,tp 1} on every engine. Proxy load_balance_proxy_server_example.py (non-layerwise, vendored) fans 12 prefill + 16 decode endpoints. DERIVED from the verified 1p1d layout by running multiple independent same-role engines; upstream documents only 1P1D, so the multi-engine fan-out is NOT separately validated. Per-engine sizing/flags identical to 1p1d. engine_id globally unique (prefill even 0/2/4, decode odd 1); kv_port stays 36000/36100 because engines are on distinct nodes. kv_port 36000/36100 deviate from upstream 30000/30100 (official table reserves [20000,36000) for AscendDirectTransport on 16-NPU nodes). Use this prefill-heavy ratio when prefill is the bottleneck (long prompts, low output length). Deviations from upstream: /root/model; NIC/IP resolved at runtime via the spine; LD_LIBRARY_PATH=/usr/local/lib for Mooncake. Treat as a starting point.", "files": [ "check_hccn.sh", "launch_online_dp.py", "load_balance_proxy_server_example.py", "meta.yaml", "run.sh", "run_decode.sh", "run_dp_template_decode.sh", "run_dp_template_prefill.sh", "run_prefill.sh", "run_proxy.sh", "setup_rank_env.sh", "smoke_test.py" ], "source_path": "models/deepseekv4-flash-a3/3p1d", "source_url": "https://github.com/xuopoj/ma-vllm-samples/tree/403c1f18cba35f9c81c75359632fc4954cecf509/models/deepseekv4-flash-a3/3p1d" } ] }, { "id": "deepseekv4-pro-a3", "title": "DeepSeek-V4-Pro · A3", "platform": "A3", "layouts": [ { "layout": "2nodes", "image": "quay.io/ascend/vllm-ascend:v0.21.0rc1-a3", "verified": false, "historical_use": false, "derived": "2026-06-27", "upstream": "https://docs.vllm.ai/projects/ascend/en/latest/tutorials/models/DeepSeek-V4-Pro.html", "note": "DeepSeek-V4-Pro-w4a8-mtp, 2x A3, TP=16 x DP=2. Already aligned with the v0.21.0rc1 Pro 2-node reference (mtp/enforce_eager). Deliberate deviation from upstream: uses --data-parallel-rank (not --data-parallel-start-rank, which causes a front-end handshake timeout per repo gotcha).", "files": [ "check_hccn.sh", "meta.yaml", "run.sh", "run_node0.sh", "run_node1.sh", "setup_rank_env.sh" ], "source_path": "models/deepseekv4-pro-a3/2nodes", "source_url": "https://github.com/xuopoj/ma-vllm-samples/tree/403c1f18cba35f9c81c75359632fc4954cecf509/models/deepseekv4-pro-a3/2nodes" } ] }, { "id": "glm5.1-a3", "title": "GLM-5.1 · A3", "platform": "A3", "layouts": [ { "layout": "1node", "image": "TODO", "verified": false, "historical_use": false, "derived": "2026-06-28", "upstream": "https://docs.vllm.ai/projects/ascend/en/latest/tutorials/models/GLM5.html", "note": "GLM-5.1-w8a8 (Eco-Tech/GLM-5.1-w8a8), single A3 node (16 NPU), DP=1 x TP=16. The GLM5.html page covers GLM-5 and GLM-5.1 with identical commands (model weights differ only); extracted the A3 w8a8 single-node block. served-model-name glm-5 (upstream; the name clients call). Speculative method deepseek_mtp is GLM-5/5.1 own MTP per upstream. Deviations: /root/model (not modelscope cache vllm-ascend/GLM5-w8a8), --port 8080 (upstream 8077). env (HCCL_*, VLLM_ASCEND_BALANCE_SCHEDULING/MLAPO/FLASHCOMM1) and serve flags kept verbatim.", "files": [ "check_hccn.sh", "meta.yaml", "run.sh" ], "source_path": "models/glm5.1-a3/1node", "source_url": "https://github.com/xuopoj/ma-vllm-samples/tree/403c1f18cba35f9c81c75359632fc4954cecf509/models/glm5.1-a3/1node" }, { "layout": "1x2p1x2d", "image": "TODO", "verified": false, "historical_use": false, "derived": "2026-06-30", "upstream": "https://docs.vllm.ai/projects/ascend/en/latest/tutorials/models/GLM5.html", "note": "GLM-5.1-w8a8 (Eco-Tech/GLM-5.1-w8a8) PD on 4x A3 (16 NPU/node): 1 prefill engine spanning 2 nodes (DP=2 x TP=16, dp-local=1, 1 instance/node, kv_producer 30000) + 1 decode engine spanning 2 nodes (DP=8 x TP=4, dp-local=4, 4 instances/node, kv_consumer 30100). This is the half-decode sibling of 1x2p1x4d: prefill side is IDENTICAL (DP=2 x TP=16), decode shrunk from 4 nodes (DP=16) to 2 nodes (DP=8); per-decode-instance sizing (TP=4, 4 NPU, max-num-seqs 8) is unchanged. External online DP (launch_online_dp.py + run_dp_template_{prefill,decode}.sh with --data-parallel-rank per instance). MooncakeConnectorV1, kv_connector_extra_config prefill{dp 2,tp 16}/decode{dp 8,tp 4} on BOTH roles, use_ascend_direct. Proxy load_balance_proxy_server_example.py (non-layerwise, vendored) fans 2 prefill + 8 decode endpoints. tool-call-parser glm47, reasoning-parser glm45, speculative deepseek_mtp. The GLM5.html page covers GLM-5/GLM-5.1 with identical commands; extracted the w8a8 PD block. Deviations from upstream: /root/model; NIC/IP + master addrs resolved at runtime via the spine; rpc-ports 10521 (prefill) / 10523 (decode) and vllm-start-ports 9081 / 9900 kept; added LD_LIBRARY_PATH=/usr/local/lib for Mooncake ascend_transport.so (ModelArts sh launch); DROPPED upstream --profiler-config (a debug artifact, not a serving flag). kv_ports per upstream. Note: prefill --max-model-len 131072 vs decode 200000 is asymmetric BY DESIGN per upstream 5.3 -- not an introduced inconsistency; do not \"fix\" to match. The smaller decode is NOT separately documented upstream (upstream shows the 4-node decode); treat as a derived starting point -- the decode DP=8 fan-out and the dp_size=8 in kv_connector_extra_config are the only changes vs the validated-shape 1x2p1x4d.", "files": [ "check_hccn.sh", "launch_online_dp.py", "load_balance_proxy_server_example.py", "meta.yaml", "run.sh", "run_decode_headless.sh", "run_decode_node0.sh", "run_dp_template_decode.sh", "run_dp_template_prefill.sh", "run_prefill_headless.sh", "run_prefill_node0.sh", "run_proxy.sh", "setup_rank_env.sh", "smoke_test.py" ], "source_path": "models/glm5.1-a3/1x2p1x2d", "source_url": "https://github.com/xuopoj/ma-vllm-samples/tree/403c1f18cba35f9c81c75359632fc4954cecf509/models/glm5.1-a3/1x2p1x2d" }, { "layout": "1x2p1x4d", "image": "TODO", "verified": false, "historical_use": false, "derived": "2026-06-28", "upstream": "https://docs.vllm.ai/projects/ascend/en/latest/tutorials/models/GLM5.html", "note": "GLM-5.1-w8a8 (Eco-Tech/GLM-5.1-w8a8) PD on 6x A3 (16 NPU/node): 1 prefill engine spanning 2 nodes (DP=2 x TP=16, dp-local=1, 1 instance/node, kv_producer 30000) + 1 decode engine spanning 4 nodes (DP=16 x TP=4, dp-local=4, 4 instances/node, kv_consumer 30100). External online DP (launch_online_dp.py + run_dp_template_{prefill,decode}.sh with --data-parallel-rank per instance). MooncakeConnectorV1, kv_connector_extra_config prefill{dp 2,tp 16}/decode{dp 16,tp 4}, use_ascend_direct. Proxy load_balance_proxy_server_example.py (non-layerwise, vendored) fans 2 prefill + 16 decode endpoints. tool-call-parser glm47, reasoning-parser glm45, speculative deepseek_mtp. The GLM5.html page covers GLM-5/GLM-5.1 with identical commands; extracted the w8a8 PD block. Deviations from upstream: /root/model; NIC/IP + master addrs resolved at runtime via the spine; rpc-ports 10521 (prefill) / 10523 (decode) and vllm-start-ports 9081 / 9900 kept; added LD_LIBRARY_PATH=/usr/local/lib for Mooncake ascend_transport.so (ModelArts sh launch); DROPPED upstream --profiler-config (forces torch profiling to ./vllm_profile on every instance -- a debug artifact, not a serving flag). kv_ports/dp-tp sizes per upstream. Note: prefill --max-model-len 131072 vs decode 200000 is asymmetric BY DESIGN per upstream 5.3 (prefill run_p.sh=131072, decode run_d*.sh=200000) -- not an introduced inconsistency; do not \"fix\" to match.", "files": [ "check_hccn.sh", "launch_online_dp.py", "load_balance_proxy_server_example.py", "meta.yaml", "run.sh", "run_decode_headless.sh", "run_decode_node0.sh", "run_dp_template_decode.sh", "run_dp_template_prefill.sh", "run_prefill_headless.sh", "run_prefill_node0.sh", "run_proxy.sh", "setup_rank_env.sh", "smoke_test.py" ], "source_path": "models/glm5.1-a3/1x2p1x4d", "source_url": "https://github.com/xuopoj/ma-vllm-samples/tree/403c1f18cba35f9c81c75359632fc4954cecf509/models/glm5.1-a3/1x2p1x4d" }, { "layout": "2nodes", "image": "TODO", "verified": false, "historical_use": false, "derived": "2026-06-28", "upstream": "https://docs.vllm.ai/projects/ascend/en/latest/tutorials/models/GLM5.html", "note": "GLM-5.1-w8a8 (Eco-Tech/GLM-5.1-w8a8), 2x A3 (16 NPU/node) single mixed engine, DP=2 x TP=16 (dp-local=1, one DP rank per node). rank0=leader (API :8080), rank1=headless. The GLM5.html page covers GLM-5/GLM-5.1 with identical commands; extracted the A3 w8a8 multi-node block. served-model-name glm-5, speculative deepseek_mtp (GLM own MTP). headless node keeps --data-parallel-start-rank (no per-node API server). Deviations: /root/model (not vllm-ascend/GLM5-w8a8), --port 8080 (upstream 8077); NIC/IP resolved at runtime via the spine (upstream had placeholder nic_name/local_ip/node0_ip).", "files": [ "check_hccn.sh", "meta.yaml", "run.sh", "run_node0.sh", "run_node1.sh", "setup_rank_env.sh" ], "source_path": "models/glm5.1-a3/2nodes", "source_url": "https://github.com/xuopoj/ma-vllm-samples/tree/403c1f18cba35f9c81c75359632fc4954cecf509/models/glm5.1-a3/2nodes" } ] }, { "id": "glm5.2-a2", "title": "GLM-5.2 · A2", "platform": "A2", "layouts": [ { "layout": "1x4p1x4d", "image": "quay.io/ascend/vllm-ascend:glm5.2", "verified": false, "historical_use": false, "derived": "2026-06-27", "upstream": "https://docs.vllm.ai/projects/ascend/en/latest/tutorials/models/GLM5.2.html", "note": ">", "files": [ "check_hccn.sh", "launch_online_dp.py", "load_balance_proxy_server_example.py", "meta.yaml", "run.sh", "run_decode_headless.sh", "run_decode_node0.sh", "run_dp_template_decode.sh", "run_dp_template_prefill.sh", "run_prefill_headless.sh", "run_prefill_node0.sh", "run_proxy.sh", "setup_rank_env.sh", "smoke_test.py" ], "source_path": "models/glm5.2-a2/1x4p1x4d", "source_url": "https://github.com/xuopoj/ma-vllm-samples/tree/403c1f18cba35f9c81c75359632fc4954cecf509/models/glm5.2-a2/1x4p1x4d" }, { "layout": "2nodes", "image": "quay.io/ascend/vllm-ascend:glm5.2", "verified": false, "historical_use": false, "derived": "2026-06-27", "upstream": "https://docs.vllm.ai/projects/ascend/en/latest/tutorials/models/GLM5.2.html", "note": "GLM-5.2-w8a8, 2x A2 (8 NPU/node) single mixed engine, DP=2 x TP=8 (dp-local=1, one DP rank per node). rank0=leader (API :8080), rank1=headless. No single-node A2 exists upstream (model needs >=2 A2 nodes). served-model-name glm-52, speculative deepseek_mtp (GLM-5.2 own MTP). Deviations: /root/model and --port 8080 (upstream cache path + 7000); upstream --max_model_len normalized to --max-model-len.", "files": [ "check_hccn.sh", "meta.yaml", "run.sh", "run_node0.sh", "run_node1.sh", "setup_rank_env.sh" ], "source_path": "models/glm5.2-a2/2nodes", "source_url": "https://github.com/xuopoj/ma-vllm-samples/tree/403c1f18cba35f9c81c75359632fc4954cecf509/models/glm5.2-a2/2nodes" } ] }, { "id": "glm5.2-a3", "title": "GLM-5.2 · A3", "platform": "A3", "layouts": [ { "layout": "1node", "image": "quay.io/ascend/vllm-ascend:glm5.2-a3", "verified": false, "historical_use": false, "derived": "2026-06-27", "upstream": "https://docs.vllm.ai/projects/ascend/en/latest/tutorials/models/GLM5.2.html", "note": "GLM-5.2-w8a8, single A3 node, DP=2 x TP=8. served-model-name glm-52 (upstream). Speculative method deepseek_mtp is GLM-5.2 own MTP per upstream. Deviations: /root/model and --port 8080 (repo conventions; upstream cache path + 8077).", "files": [ "check_hccn.sh", "meta.yaml", "run.sh" ], "source_path": "models/glm5.2-a3/1node", "source_url": "https://github.com/xuopoj/ma-vllm-samples/tree/403c1f18cba35f9c81c75359632fc4954cecf509/models/glm5.2-a3/1node" }, { "layout": "1x2p1x2d", "image": "quay.io/ascend/vllm-ascend:glm5.2-a3", "verified": false, "historical_use": false, "derived": "2026-06-27", "upstream": "https://docs.vllm.ai/projects/ascend/en/latest/tutorials/models/GLM5.2.html", "note": ">", "files": [ "check_hccn.sh", "launch_online_dp.py", "load_balance_proxy_server_example.py", "meta.yaml", "run.sh", "run_decode_node0.sh", "run_decode_node1.sh", "run_dp_template_decode.sh", "run_dp_template_prefill.sh", "run_prefill_node0.sh", "run_prefill_node1.sh", "run_proxy.sh", "setup_rank_env.sh", "smoke_test.py" ], "source_path": "models/glm5.2-a3/1x2p1x2d", "source_url": "https://github.com/xuopoj/ma-vllm-samples/tree/403c1f18cba35f9c81c75359632fc4954cecf509/models/glm5.2-a3/1x2p1x2d" }, { "layout": "2nodes", "image": "quay.io/ascend/vllm-ascend:glm5.2-a3", "verified": false, "historical_use": false, "derived": "2026-06-27", "upstream": "https://docs.vllm.ai/projects/ascend/en/latest/tutorials/models/GLM5.2.html", "note": "GLM-5.2-w8a8, 2x A3 single mixed engine, DP=2 x TP=16 (dp-local=1, one DP rank per node). rank0=leader (API :8080), rank1=headless. served-model-name glm-52, speculative method deepseek_mtp (GLM-5.2 own MTP). Deviations: /root/model and --port 8080 (upstream cache path + 8077).", "files": [ "check_hccn.sh", "meta.yaml", "run.sh", "run_node0.sh", "run_node1.sh", "setup_rank_env.sh" ], "source_path": "models/glm5.2-a3/2nodes", "source_url": "https://github.com/xuopoj/ma-vllm-samples/tree/403c1f18cba35f9c81c75359632fc4954cecf509/models/glm5.2-a3/2nodes" }, { "layout": "4nodes", "image": "quay.io/ascend/vllm-ascend:glm5.2-a3", "verified": false, "historical_use": false, "derived": "2026-06-27", "upstream": "https://docs.vllm.ai/projects/ascend/en/latest/tutorials/models/GLM5.2.html", "note": "GLM-5.2-w8a8, 4x A3 co-located mixed engine for 200k context, DP=4 x TP=8 (dp-local=1, one DP rank per node). rank0=leader (API :8080), ranks1-3=headless via a single run_node_headless.sh (dp-start-rank == node rank). served-model-name glm-52, speculative deepseek_mtp. Deviations: /root/model and --port 8080 (upstream cache path + 7000); upstream --max_model_len normalized to --max-model-len. NOTE: dp-local 1 x tp 8 = 8 uses only 8 of the 16 NPUs/node -- this is verbatim from the upstream 200k co-located config (it trades NPU count for context length), not a TP transcription slip; do not \"fix\" to TP=16.", "files": [ "check_hccn.sh", "meta.yaml", "run.sh", "run_node0.sh", "run_node_headless.sh", "setup_rank_env.sh" ], "source_path": "models/glm5.2-a3/4nodes", "source_url": "https://github.com/xuopoj/ma-vllm-samples/tree/403c1f18cba35f9c81c75359632fc4954cecf509/models/glm5.2-a3/4nodes" } ] }, { "id": "qwen3.5-397b-a17b-a2", "title": "Qwen3.5-397B-A17B · A2", "platform": "A2", "layouts": [ { "layout": "2nodes", "image": "TODO", "verified": false, "historical_use": false, "derived": "2026-06-28", "upstream": "https://docs.vllm.ai/projects/ascend/en/latest/tutorials/models/Qwen3.5-397B-A17B.html", "note": "Qwen3.5-397B-A17B-w8a8-mtp, 2x A2 (8 NPU/node) single mixed engine, DP=2 x TP=8 (dp-local=1, one DP rank per node). rank0=leader (API :8080, --api-server-count 2), rank1=headless. served-model-name qwen3.5, speculative qwen3_5_mtp (Qwen3.5 own MTP). headless node keeps --data-parallel-start-rank (no per-node API server, so --data-parallel-rank not needed). Deviations from upstream 5.2: /root/model, --port 8080 (upstream 8000), dropped VLLM_USE_MODELSCOPE; NIC/IP resolved at runtime via the spine (upstream had placeholder nic_name/local_ip/node0_ip).", "files": [ "check_hccn.sh", "meta.yaml", "run.sh", "run_node0.sh", "run_node1.sh", "setup_rank_env.sh" ], "source_path": "models/qwen3.5-397b-a17b-a2/2nodes", "source_url": "https://github.com/xuopoj/ma-vllm-samples/tree/403c1f18cba35f9c81c75359632fc4954cecf509/models/qwen3.5-397b-a17b-a2/2nodes" } ] }, { "id": "qwen3.5-397b-a17b-a3", "title": "Qwen3.5-397B-A17B · A3", "platform": "A3", "layouts": [ { "layout": "1node", "image": "TODO", "verified": false, "historical_use": false, "derived": "2026-06-28", "upstream": "https://docs.vllm.ai/projects/ascend/en/latest/tutorials/models/Qwen3.5-397B-A17B.html", "note": "Qwen3.5-397B-A17B-w8a8-mtp, single A3 node (16 NPU), DP=1 x TP=16. served-model-name qwen3.5 (upstream). Speculative method qwen3_5_mtp is Qwen3.5 own MTP per upstream. Deviations from upstream 5.1: /root/model (not modelscope cache Eco-Tech/Qwen3.5-397B-A17B-w8a8-mtp), --port 8080 (upstream 8000), dropped VLLM_USE_MODELSCOPE (model pre-mounted). All other env (HCCL_*, jemalloc LD_PRELOAD, cpufreq/sysctl tuning) and serve flags kept verbatim.", "files": [ "check_hccn.sh", "meta.yaml", "run.sh" ], "source_path": "models/qwen3.5-397b-a17b-a3/1node", "source_url": "https://github.com/xuopoj/ma-vllm-samples/tree/403c1f18cba35f9c81c75359632fc4954cecf509/models/qwen3.5-397b-a17b-a3/1node" }, { "layout": "1p1x2d", "image": "TODO", "verified": false, "historical_use": false, "derived": "2026-06-28", "upstream": "https://docs.vllm.ai/projects/ascend/en/latest/tutorials/models/Qwen3.5-397B-A17B.html", "note": "Qwen3.5-397B-A17B-w8a8-mtp PD: 1 single-node prefill (DP=8 x TP=2, kv_producer, kv_port 23010) + 1 decode spanning 2 A3 nodes (DP=16 x TP=2, kv_consumer, kv_port 36010). MooncakeLayerwiseConnector (NOT MooncakeConnectorV1) with kv_connector_extra_config prefill{dp 8,tp 2}/decode{dp 16,tp 2}; proxy is load_balance_proxy_layerwise_server_example.py (layerwise variant, vendored from vllm-ascend main; falls back to the image copy). rank0=prefill, rank1=decode master (API), rank2=decode headless. served-model-name qwen3.5, speculative qwen3_5_mtp. Deviations from upstream 5.4: /root/model; engine API ports unified to 7100 (upstream 30060/30050) so proxy wiring derives from AISHIPBOX_ADDRS; NIC/IP resolved at runtime via the spine; added LD_LIBRARY_PATH=/usr/local/lib for Mooncake ascend_transport.so (ModelArts sh launch). kv_port values, connector type, dp/tp sizes, env block kept per upstream. KNOWN RISK: the decode master (run_decode_node0.sh) runs an API server (--api-server-count 1) yet uses --data-parallel-start-rank 0, which CLAUDE.md/[[dp-rank-vs-start-rank]] warns can cause a ~5-min front-end handshake timeout for per-node-API-server multi-node DP. Kept as-is because it is upstream-faithful (Qwen 5.4 run_d0.sh uses start-rank 0); the verified deepseek 2p1x2d decode master passes NO start-rank flag instead. If decode never becomes ready on bring-up, try dropping --data-parallel-start-rank 0 on the master (keep it only on the headless worker run_decode_node1.sh).", "files": [ "check_hccn.sh", "load_balance_proxy_layerwise_server_example.py", "meta.yaml", "run.sh", "run_decode_node0.sh", "run_decode_node1.sh", "run_prefill_node0.sh", "run_proxy.sh", "setup_rank_env.sh", "smoke_test.py" ], "source_path": "models/qwen3.5-397b-a17b-a3/1p1x2d", "source_url": "https://github.com/xuopoj/ma-vllm-samples/tree/403c1f18cba35f9c81c75359632fc4954cecf509/models/qwen3.5-397b-a17b-a3/1p1x2d" } ] } ] }