Per node
128 GB
LPDDR5x-8000, unified in-node
AI technical datasheet · revised July 2026
The inference cluster, in detail.
How four 128 GB AMD Ryzen AI Max+ 395 nodes are engineered into one coordinated distributed-inference system. Every figure states its operating conditions — deliberately. Numbers without conditions are marketing; numbers with conditions are engineering.
512 GB aggregate
Ubuntu · ROCm · llama.cpp
Conditions on every figure
Local BW
≈256 GB/s
per node — 25–40× the fabric
Runtime
llama.cpp RPC
isolated private network
01 · Node platform
One node, specified from the vendor sheet.
Per-node figures follow AMD's published processor specifications. System-level I/O — in particular USB4 v2 / 80 Gbps ports — is a vendor-board property, so we verify it per chassis rather than inheriting it from the CPU page.
Per-node specificationAMD published data
CPU16 cores / 32 threadsZen 5 · native 512-bit AVX-5126
Integrated GPURadeon 8060S40 compute units
Memory128 GB LPDDR5x-8000256-bit unified CPU+GPU · ≈96 GB (Windows) / 110–120 GB (Linux) GPU-allocatable6
Local memory BW≈ 256 GB/s8000 MT/s × 256-bit ÷ 8
ExpansionPCIe 4.0
USB42 × 40 Gbps native80 Gbps USB4 v2 depends on the vendor board & controller2
Four-node aggregateacross nodes — not a shared pool
Memory512 GB across four nodes128 GB × 4 — scales proportionally with node count1
CPU64 cores / 128 threads
GPU160 compute units aggregate
Local memory BW4 × ≈ 256 GB/seach node accesses only its own memory at this rate
Runtimecoordinated distributed inferencethe model is sharded across nodes, not one VRAM pool
02 · Bandwidth reality
Local memory is 25–40× faster than any external fabric.
This single ratio drives the whole architecture. The runtime keeps tensors home and crosses only when it must — which is also why adding fabric bandwidth never buys linear tokens/s, and why we engineer topology to the model's traffic pattern instead of quoting a headline link speed.
per-node local memory = 8000 MT/s × 256-bit ÷ 8 ≈ 256 GB/s
USB4 v2 raw link = 80 Gbps = 10 GB/s
dual 25GbE aggregate = 2 × 25 Gbps = 6.25 GB/s
Linear scale, theoretical figures — the gap is the point. Distributed inference works here because llama.cpp RPC shards layers so the heaviest traffic stays inside each node.
03 · Interconnect fabrics
Three fabrics, engineered — not just cabled.
USB4 v2 for cost, Dual 25GbE for repeatable four-node operation, hybrid multi-path for maximum headroom on selected pairs. Each carries its topology, its strengths and its honest constraints.
Option A · entry fabric
USB4 v2 direct-link
USB4 host-to-host networking (USB4NET) lets two systems joined by a certified cable talk like Ethernet peers — no NIC, no switch. Ideal for a two-node pilot; the cheapest path to private local AI.
Constraint, plainly: USB4 is point-to-point, not switched. Four-node USB4-only builds need chain or ring designs and ship only after the routes are measured. Host-to-host support depends on the vendor board, controller, cable, firmware and OS.2
Option B · professional fabric · recommended
PCIe Dual 25GbE fabric
An ENPBA-class PCIe card per node — Intel E810-class controller, two 25GbE SFP28 ports and two M.2 NVMe slots — through a compact SFP28 switch. Standard Ethernet means standard tooling: bonding, MTU tuning, per-port monitoring. The recommended default for a production ArrayMeld 4.
Aggregate means aggregate. Two 25GbE links give up to 50 Gbps per node across flows; a single stream rides one 25 Gbps path unless MPTCP, multi-flow traffic or bonding is configured — engineering we perform and verify with iperf3.3
Option C · maximum headroom
Hybrid multi-path
The full 25GbE switch backbone still reaches every node — so the controller talks to all workers — and the controller adds direct USB4 links to accelerate its heaviest worker paths. Each board exposes two USB4 ports, so the controller can direct-link at most two workers; any remaining node rides the fabric. Reserved for the traffic patterns that justify the extra engineering, and only there.
Raw paths ≠ throughput. Applications do not automatically use the summed capacity of every interface. Multi-path aggregation (MPTCP or policy routing) must be engineered and measured per topology, and is bounded by controller uplinks and the software stack. Sold as a validated design, never as an automatic number.4
Why not a faster single NIC? The platform's expansion slot is PCIe 4.0 ×4 electrical, whose realistic payload (≈6.7–7.1 GB/s) is almost perfectly matched by dual 25GbE (6.25 GB/s) — so the recommended card already uses ~90% of what the slot can physically move. A 40/50/100G NIC is a legitimate single-stream upgrade offered as custom validated engineering, not a default — and all of it is dwarfed by the ≈256 GB/s of local memory, which is why more fabric bandwidth does not buy linear tokens/s.3
04 · Software stack
The stack AMD's own cluster guide runs — tuned by us.
Linux is the serious path for multi-node work: full control of interfaces, MPTCP, routing and kernel parameters, and the best ROCm alignment for this platform. Each release records its exact versions and is regression-tested before it becomes supported.
Agent toolsHermes · OpenClaw · Open WebUI · IDE coding agents — anything speaking the OpenAI-compatible API
Model serverLocal OpenAI-compatible endpoint on the controller node
llama.cpp + RPC · ROCm/HIPDistributed runtime; controller shards the model to rpc-server workers on GPU (gfx1151)
Ubuntu 24.04 LTSMPTCP stack, ice driver for E810-class NICs, container provisioning, diagnostics
4 × AI Max+ 395128 GB per node · USB4 v2 / Dual 25GbE / hybrid fabric
# build llama.cpp with HIP + RPC (per node) cmake -B rocm -DGGML_HIP=ON -DGGML_RPC=ON \ -DAMDGPU_TARGETS="gfx1151" cmake --build rocm --config Release -j$(nproc) # workers — isolated private segment only ./rpc-server -p 50053 -c --host 0.0.0.0 # controller — GLM-5.2 across three workers ./llama-server -m GLM-5.2-Q4.gguf -c 32768 \ -fa on -ngl 999 --port 8080 \ --rpc 10.25.0.2:50053,10.25.0.3:50053,10.25.0.4:50053
RPC security discipline. Upstream documentation describes the llama.cpp RPC backend as proof-of-concept and insecure on open networks. Our deployment confines RPC to an isolated private segment, firewall-restricted, never internet-exposed, soak-tested for stability before delivery.5
05 · Model sizing — GLM-5.2
Why two nodes is the floor and four is the recommendation.
Model fit is arithmetic before it is anything else: weights plus KV cache plus runtime overhead must live inside aggregate node memory, with headroom. The ladder below is sized from public GGUF figures, then verified on the real system.
| Deployment target | Nodes / aggregate | On-disk weights | Positioning |
|---|---|---|---|
| GLM-5.2 Q2 / 2-bit GGUF | 2 × 128 GB — 256 GB | ≈ 238–254 GB | Entry / validation — deliberately tight; context budget controlled and measured |
| GLM-5.2 4-bit / W4-class | 4 × 128 GB — 512 GB | ≈ 365 GB+ | Recommended production coding-agent target — headroom for KV cache, context and runtime |
| 8-node dual-fabric | 8 × 128 GB — 1,024 GB | ≈ 700–900 GB class | Scale-out custom engineering — capacity, precision and context scale; per-token latency does not |
| Larger / future open models | 4+ nodes or custom | Model-specific | Custom engineering with model-specific validation before commitment |
Proven pattern. AMD's published Ryzen AI Max+ cluster guide runs a one-trillion-parameter-class quantised model (≈375 GB) across four 128 GB systems on Ubuntu 24.04, ROCm and llama.cpp RPC over Ethernet. Our recommended tier is the same architecture, engineered and validated as a product.
The cost case. Reaching ≈512 GB-class local capacity this way costs about AUD 20,000 of standard hardware — roughly a third to a fifth of the multi-GPU, Mac-fleet or DGX-class routes (see the cost comparison). Token throughput is not the highest available; the decisive value for a cost-sensitive team is running 400B-parameter-class models at all, on owned hardware, within a small-team budget.*
Model caveat. GLM-5.2 — with its 1M-token context target and strong coding positioning in public documentation — is our current recommendation, not a permanent guarantee. Open models evolve; the cluster redeploys. Qwen 3.6 and Gemma 4 are strong single-node alternatives deployed on request; proprietary API-only tiers cannot run locally.5
06 · Validation
Every cluster leaves with numbers attached.
We do not publish tokens/s until it is measured on the final hardware, model, quantisation, context length, runtime version and topology — and neither should anyone else. Here is what gets measured.
Two-node validation (entry tier)
- USB4NET single-path and dual-link iperf3, then MPTCP-aggregated
- 25GbE per-port and multi-flow iperf3
- llama.cpp RPC over single path, then over MPTCP
- GLM-5.2 Q2 load, time-to-first-token, tokens/s, stability
- Memory, temperature and power monitored throughout
Four-node validation (recommended tier)
- All-node 25GbE validation; jumbo frames / LACP / MPTCP where used
- llama.cpp RPC with 2, 3 and 4 workers
- GLM-5.2 4-bit-class model loading and long-context stability
- Coding-agent workflow test — Hermes or OpenClaw
- Recovery after worker restart; NVMe staging performance
Model load timeTime to first tokenGeneration tok/sPrompt tok/s
Network per pathPer-node memoryThermalsMulti-hour stability
07 · Specification discipline
Our claim matrix, published on purpose.
An audience that reads spec sheets deserves a vendor that writes them honestly. These are the wording rules we hold ourselves to — on this site, in quotes, and in person.
| Topic | We say | We won't say |
|---|---|---|
| Memory | 512 GB aggregate memory-class capacity across four 128 GB nodes | "512 GB unified VRAM" |
| Model scale | Designed for large low-bit open models, subject to validation | "Guaranteed to run every 400B model" |
| GLM-5.2 | Current recommended target as of July 2026 | "Permanently the best model" |
| USB4 v2 | Up to 80 Gbps raw direct-link capability where the platform supports it | "Guaranteed 80 Gbps application throughput" |
| Dual 25GbE | Up to 50 Gbps aggregate capacity per node | "Every stream runs at 50 Gbps" |
| Hybrid fabric | 100 Gbps+ class multi-path on selected pairs, subject to validation | "Automatic 130 Gbps connection" |
| Vs closed models | Strong enough for many practical local coding-agent workflows | "Equal or superior to closed frontier models" — unless our benchmarks show it on your workload |
08 · Reference sources
Where the figures come from.
Hardware, pricing, model rankings and software support all move. We re-check these before publishing final claims — and we recommend you click through and do the same.
AMD Ryzen AI Max+ 395
16 Zen 5 cores, 128 GB max, 256-bit LPDDR5x-8000, Radeon 8060S (40 CUs), PCIe 4.0, native USB4 listing.
amd.com — product pageAMD four-node cluster guide
Four 128 GB Ryzen AI Max+ 395 systems, Ubuntu 24.04, ROCm and llama.cpp RPC running a trillion-parameter-class quantised model over Ethernet.
amd.com — technical articleAMD ROCm — Ryzen Linux matrix
Supported gfx1151 / Ubuntu / ROCm combinations and validated data types.
rocm.docs.amd.comMicrosoft USB4NET
USB4 interdomain networking: two USB4 PCs joined by a cable enumerate a network adapter and communicate like Ethernet peers.
learn.microsoft.comLinux kernel MPTCP
MPTCP can use multiple interfaces within a single connection, aggregating bandwidth or preferring lower-latency paths.
docs.kernel.orgllama.cpp RPC backend
rpc-server exposes remote devices for distributed inference; upstream flags the backend proof-of-concept and insecure on open networks.
github.com — llama.cppA source link does not freeze a specification. Before we rely on any figure for a contract or claim, we open the current official source, record the date, and validate it on the delivered system.
From datasheet to delivered system
Bring your target model. We'll design the cluster around it.
Send your target model, context requirements and budget path — we'll return a scoped topology, a bill of materials, and the validation plan we would sign off against.