SILICON TALES

Advisory · AI infrastructure strategy

The AI factory blueprint: match the silicon to the strategy

Anyone can read a spec sheet. The harder question is which NVIDIA system your strategy actually needs, and at which numeric precision.

Silicon Tales Blueprint 15 July 2026 18 min read

Jensen Huang stood on the CES 2026 stage, held up a Vera Rubin board, and told the room it was already in full production. The promise attached to it was the kind that makes finance directors reach for the chequebook: up to five times the inference performance of Blackwell and roughly a tenth of the cost per token. Somewhere in every audience for that pitch sits a person whose real job is to ask a quieter question. Not "how fast is it," but "fast at what, for whom, and paid back over how many years."

This piece is written for that person. It is a blueprint, not a hype reel. The tools have never been more powerful, and the temptation to buy the biggest box in the catalogue has never been stronger, which is exactly why the discipline matters. The right platform is not the fastest one. It is the one your operating model actually needs. So we start where a good advisor always starts, with the work, and only then do we reach for the silicon.

96 → 288
GB of memory per GPU, RTX PRO to Blackwell Ultra and Rubin
1 → 72
GPUs inside one NVLink scale-up domain, PCIe island to full rack
FP4 → FP64
Six numeric precisions, each with a different job
142 kW
Up to, per NVL72 rack. The number your building has to answer
Advertisement

01/ Strategy before silicon

The most expensive mistake is buying capability you cannot feed

There is a pattern that plays out in procurement meetings all over the world right now. A vendor deck lands, the headline multiplier is enormous, and the conversation jumps straight to which rack to order. The operating model, the actual shape of the work the organisation does, never enters the room. That is the moment a good advisor slows everything down, because the fastest way to waste a seven figure budget is to buy a machine your workload can never keep busy.

The discipline is old and it is boring and it works. Look honestly at your current operating model: what you run today, how often, at what latency, for how many users, against models of what size. Then define the target operating model: where you intend to be in eighteen to thirty six months, not where a keynote says the industry is going. Only when those two pictures are clear do you choose technology, because now you are matching a system to a purpose instead of buying a purpose to justify a system. Strategy first, silicon second, in that order every time.

None of this is an argument against ambition or against NVIDIA. The point is the opposite. When you buy against a real target operating model, you tend to buy more confidently and defend the spend more easily, because every line in the bill of materials traces back to a use case and a payback. The waste is not in spending money on GPUs. The waste is in spending it on the wrong axis, on raw compute when your bottleneck was latency, or on a rack of coherent memory when your models fit comfortably on a single card.

Buy the operating model, not the GPU. The right platform is the one your work will actually keep busy.

There is a second reason to start from the work rather than the chip, and it is the quiet thing NVIDIA sells that never shows up as a number on the board. When Jensen Huang bet the company on CUDA in the late 2000s, long before there was an obvious market for it, he was not selling silicon. He was building a software moat that today means your engineers, your frameworks, and your operators already speak the platform's language. That lock-in is real and it cuts both ways, so factor it into the strategy deliberately rather than discovering it after the purchase order clears.

The three ways buyers get it wrong Feeding failure: the accelerators sit idle because the data pipeline, the network, or the batch size cannot keep them fed, so utilisation and payback both collapse. Axis failure: the workload needed low latency, and the budget bought raw throughput instead, or the reverse. Facility failure: the rack was the right choice on paper, but the building could not deliver the power or the liquid cooling, and the plan stalled at the loading dock. Each of these is a strategy error wearing a hardware costume.

02/ The three questions that actually decide it

Latency, raw compute, or memory bandwidth. Pick your bottleneck

Almost every infrastructure decision collapses into one honest question: what is the thing you are actually short of? Not what sounds impressive, but what limits the work. In practice there are three answers, and a fourth that quietly gates all of them. Naming yours correctly is most of the job.

Ultra-low latency is the need when a human or an agent is waiting on every token, and the time to first token and the gap between tokens decide whether the product feels alive. Raw compute, the pure throughput of matrix math, is the need when you are training, fine tuning, or pushing enormous batches through inference and you are paid by total work done per hour. Memory bandwidth is the need that hides in plain sight: for many inference and scientific workloads the GPU is starved not for arithmetic but for the speed at which it can move weights and activations in and out of memory, and that is where the generational leaps quietly land.

The fourth dimension is working-set reach, and it decides the others. A model and its context that fit inside one GPU's memory are a different engineering problem from one that must be sharded across eight GPUs, which is different again from one that wants a coherent domain of seventy two. Reach is what turns a chip decision into a rack decision. Read the diagram below as a set of dials rather than a scoreboard: your job is to know which dial your work is turning hardest.

Latency need batch real‑time Raw compute modest extreme Memory bandwidth low very high Working‑set reach one GPU rack‑coherent RTX PRO 6000 HGX B300 GB300 NVL72 Vera Rubin NVL72 Groq 3 LPU
Figure 1. Where the platforms sit on the four dials that decide a deployment. RTX PRO is the flexible generalist, HGX B300 adds an eight-GPU scale-up island, GB300 NVL72 turns a whole rack into one memory-rich domain, and the Rubin-era Groq LPU is built almost entirely around latency. No platform wins every dial, which is exactly why the question comes before the product. Alt text: four horizontal axes labelled latency sensitivity, raw compute, memory bandwidth and working-set reach, with the five platforms placed along each.
Advertisement

03/ Precision is a strategy decision, not a footnote

The same GPU is six different machines, depending on the number format

Here is the part that most buying conversations skip, and it is the part that can flip the whole decision. A modern NVIDIA GPU does not have one performance figure. It has a ladder of them, one per numeric precision, and the rungs are separated by more than a thousand times. Choosing the precision your workload runs at is not a tuning detail you hand to the engineers afterwards. It is part of defining the workload itself, and it can change which platform you should buy.

The logic is simple once you see it. Fewer bits per number means more numbers moved and multiplied per second, and a smaller memory footprint, at the cost of numerical range and accuracy. So very low precision such as NVFP4 is the language of high-throughput inference, where a well-quantised model gives you the most tokens per second per dollar. FP8 and FP6 are the modern mainstream for training and heavy inference. FP16 and BF16 remain the numerically safe choice for training that has to converge cleanly. And FP32 and FP64 are where scientific computing, simulation, and anything that cannot tolerate rounding error still live. A platform that is breathtaking at FP4 inference can be the wrong tool for FP64 double-precision HPC, and no amount of headline PFLOPS changes that.

The Rubin GPU makes the ladder unusually stark, because NVIDIA publishes the whole thing. Read the rungs below on a logarithmic scale, because a linear one would flatten everything below FP8 into an invisible sliver. That visual crush is the point: the machine you are buying is enormous for low-precision AI and comparatively modest for double precision, so your precision has to match your purpose before the numbers mean anything.

10 TF 100 TF 1 PF 10 PF 100 PF NVFP4inference 50 PF* NVFP4training 35 PF FP8 / FP6training 17.5 PF FP16 / BF16training 4 PF TF32tensor 2 PF FP32general 130 TF FP64HPC 33 TF
Figure 2. One Rubin GPU, seven precision modes, plotted on a logarithmic axis of peak throughput. NVFP4 inference is a sparse specification, so treat it as a ceiling, not a guarantee. The gap from 50 PFLOPS of FP4 down to 33 TFLOPS of FP64 is roughly 1,500 times, which is why the number format is a strategic choice and not a checkbox. Alt text: horizontal log-scale bar chart of Rubin GPU throughput from NVFP4 at 50 PFLOPS down to FP64 at 33 TFLOPS.
Table 1. Rubin GPU peak performance by precision, per GPU, with the job each format is really for. Source: NVIDIA HGX and Rubin preliminary specifications, July 2026.
PrecisionRubin peak (per GPU)What it is really forWatch-out
NVFP4 (FP4)50 PF inference
35 PF training
Highest-throughput inference; frontier low-precision trainingInference figure is sparse; needs careful quantisation
FP8 / FP617.5 PF trainingMainstream training and high-throughput inferenceThe current sweet spot; validate accuracy per model
FP16 / BF164 PFNumerically safe training and fine-tuningFour times below FP8; use where convergence needs it
TF322 PFDrop-in tensor math for FP32-style trainingConvenience precision, not full FP32 range
FP32130 TFGeneral compute, graphics, non-tensor pathsOrders of magnitude below the tensor formats
FP6433 TFDouble-precision scientific and HPC workIf this is your need, raw AI PFLOPS mislead you
Read the asterisk NVIDIA's largest inference figure, the 50 PFLOPS NVFP4 number, is a sparse specification, meaning it assumes structured sparsity in the model. Dense workloads see less. This is not a criticism, it is how the industry quotes these parts, but a blueprint that treats a sparse "up to" ceiling as a guaranteed floor will overpromise. Size against dense numbers, then let sparsity be upside.

04/ The three factories you can buy today

One is a generalist, one is a scale-up island, one is a rack that thinks as a whole

NVIDIA's current enterprise reference architectures give you three genuinely different machines, and the difference that matters most is not on the spec sheet's first line. It is how the GPUs talk to each other. Get that one idea straight and most of the decision resolves itself, because the interconnect is what decides which models you can run without fighting your own hardware.

The RTX PRO 6000 AI factory is the flexible generalist. Its GPUs each hold 96 GB of GDDR7 and connect through PCIe, with no NVSwitch fabric binding them together, so eight cards in a chassis are eight cooperating islands rather than one pooled brain. That sounds like a limitation and sometimes it is, but it is also why this platform fits a normal enterprise rack, sips power by comparison, runs graphics and rendering and virtual workstations, and covers the widest spread of workloads of the three. The HGX B300 AI factory raises the ceiling by wiring eight Blackwell Ultra GPUs into a single NVLink domain through NVSwitch, so those eight cards, with 288 GB of fast HBM3e each, behave like one tightly coupled accelerator for training and large-model inference. The GB300 NVL72 takes the same idea and stretches it across an entire rack, so all seventy two GPUs sit inside one NVLink domain with coherent Grace CPU memory, at the cost of mandatory liquid cooling and up to 142 kilowatts that your building has to supply.

RTX PRO 6000 PCIe islands PCIe Gen5 · CPU 96 GB per GPU · no NVLink scales out over Ethernet HGX B300 8-GPU NVLink island NVSwitch 288 GB · 1.8 TB/s per GPU one 8-GPU domain GB300 NVL72 72-GPU NVLink domain one NVLink 5 fabric · 130 TB/s 72 GPUs · 36 Grace CPUs · 20 TB liquid cooled, up to 142 kW
Figure 3. The one distinction that drives the decision: the scale-up domain. RTX PRO leaves each GPU on its own PCIe island, HGX B300 fuses eight into one NVLink domain, and GB300 NVL72 fuses seventy two. A model whose working set exceeds a single card cares enormously about which of these you bought. Alt text: three schematic panels contrasting PCIe-connected GPUs, an eight-GPU NVLink island, and a seventy-two-GPU NVLink rack.
Table 2. The three NVIDIA AI factories side by side. Green marks show where each platform leads on that row, and no platform leads on every row. Source: NVIDIA enterprise reference architectures and product pages, July 2026.
Decision pointRTX PRO 6000HGX B300GB300 NVL72
GPU classRTX PRO 6000 Blackwell, PCIeB300 Blackwell Ultra, SXMB300 in Grace Blackwell Ultra
Memory per GPU96 GB GDDR7288 GB HBM3e288 GB HBM3e
Bandwidth per GPU1.6 TB/s8 TB/s8 TB/s
Scale-up fabricPCIe Gen5, no NVSwitchNVLink 5, 8-GPU domainNVLink 5, 72-GPU domain
Coherent CPU-GPU memoryNoNo (x86 host)Yes, Grace NVLink-C2C
Max GPU power (TGP)up to 600 Wup to 1,400 Wup to 1,400 W
CoolingAir or liquidAir or liquid, denseDirect liquid, mandatory
Power envelopeOEM-defined, modest~14.5 kW / systemup to 142 kW / rack
Server customisationHigh (2/4/8 GPU, CPU, RAM)Medium-highLow inside the rack
MIG partitions per GPUup to 4up to 7up to 7
Best whenMixed enterprise, graphics, small and medium trainingSerious 8-GPU training and inferenceTrillion-parameter, MoE, reasoning at rack scale

What "unified memory" actually means, and why it can cost you

The most expensive misunderstanding in this whole category is the word "unified." A datasheet that says a rack has 20 terabytes of GPU memory is telling the truth about capacity and quietly misleading you about architecture. That 20 terabytes is seventy two separate pools of 288 GB, each one physically welded to its own GPU. NVLink lets a GPU reach into a neighbour's memory quickly, and on NVL72 the Grace CPU and its GPUs share a coherent address space, but coherent addressability is not the same thing as one flat pool where every byte is equally fast.

This matters because it changes how a model has to be written and placed. On the RTX PRO platform, an eight-GPU node holds 768 GB of VRAM in total, yet a single un-sharded process cannot treat it as one 768 GB device: the model must be partitioned across the cards deliberately. NVL72 gives you a far larger and genuinely coherent domain, which is exactly why it exists, but even there local HBM is fastest, the Grace LPDDR5X is a separate slower tier, and remote GPU memory travels across NVLink. Buy the coherent rack when your working set truly needs it. Do not pay for it to solve a problem that careful sharding on a cheaper platform would have solved for a fraction of the cost.

The distinction that saves budgets Separate four ideas that marketing collapses into one word. Physical ownership: which chip the memory is bolted to. Peer access: whether another GPU can reach it quickly over NVLink. Coherent addressability: whether CPU and GPU share one address space without explicit copies. CUDA Unified Memory: a software convenience, not a promise that the bytes live on one device. Only NVL72 gives you native Grace CPU and GPU coherency, and even it keeps HBM and LPDDR5X as distinct tiers.
Advertisement

05/ The fit finder

Stop reading the table. Drag your requirements instead

A table tells you what each platform is. It does not tell you what you need. So here is the same logic turned into a tool you can steer. Set your objective, pick the precision your models really run at, then drag the working set, the latency target, the throughput you are chasing, and, crucially, how ready your building actually is. The recommendation updates as you move, and it shows its reasoning, because a recommendation you cannot interrogate is worth nothing.

Watch how the answer moves. Push latency to real-time and the whole picture tilts toward the deterministic decode engines. Switch precision from FP4 to FP32 and platforms reshuffle, because a machine tuned for low-precision inference is not the machine you want for double-precision science. Turn facility readiness down to a standard enterprise room and the biggest racks quietly disqualify themselves, no matter how good their numbers look. That last move is the one most decks never show you.

Fit finder

Drag your requirements. The recommendation updates live.

8 GB400 GB40 TB
batch, 600 msreal-time, 5 ms
100 tok/s1M tok/s
standard airDC liquid, 140 kW
Recommended for your inputs
Adjust the controls to see a match.
Fit scores are a transparent heuristic over NVIDIA's published reference specs, weighted by the inputs you set. They are a starting point for a conversation with a solutions architect, not a procurement guarantee.
Advertisement

06/ Reading the roadmap without buying the hype

Vera Rubin is not one faster chip. It is a system that stopped pretending one chip should do everything

When NVIDIA unveiled Vera Rubin, the easy headline was the raw Rubin GPU number. The more interesting story, and the one that maps directly onto everything above, is that the platform is deliberately seven specialised chips instead of one heroic accelerator. That is the operating-model-first idea rendered in silicon: stop forcing a single processor to do request handling, training, decode, networking, storage, and security, and instead give each job the chip that removes its specific bottleneck.

The cast divides cleanly. Three chips execute work: the Vera CPU, an 88-core Arm processor with up to 1.5 TB of memory for orchestration, agents, and KV-cache management; the Rubin GPU, with 288 GB of HBM4 and the broad training and inference role; and the Groq 3 LPU, an SRAM-heavy accelerator built for one thing, deterministic low-latency decode. The other four move, secure, and coordinate data: NVLink 6 for GPU scale-up, ConnectX-9 and Spectrum-6 for scale-out, and BlueField-4 for infrastructure, storage, and in-silicon security. Nobody is asking you to buy all seven in a box. You compose the ones your target operating model needs.

Vera Rubin platform seven chips, one composable system EXECUTION Vera CPU 88 Arm cores · up to 1.5 TB · agents, ETL, KV Rubin GPU 288 GB HBM4 · up to 22 TB/s · train, prefill Groq 3 LPU 500 MB SRAM · 150 TB/s · low-latency decode SCALE-UP NVLink 6 scale-up switch 3.6 TB/s / GPU 260 TB/s / rack SHARP FP8 in-network reduce SCALE-OUT ConnectX-9 SuperNIC endpoint 1.6 Tb/s / GPU · RDMA Spectrum-6 CPO Ethernet switch 102.4 Tb/s · optics INFRA BlueField-4 DPU 64 Arm cores 128 GB memory security offload context memory
Figure 4. The Vera Rubin platform grouped by the job each chip does. Only three of the seven run application or model code. The rest exist so those three are never waiting on the network, the storage layer, or the security stack. Alt text: seven NVIDIA chips organised into four groups, execution, scale-up, scale-out and infrastructure. Source: NVIDIA Vera Rubin technical materials, July 2026, preliminary.

The three engines that actually compute, and why they coexist

Look closely at the three execution chips and you see the precision and bandwidth arguments from earlier turned into hardware. The numbers that matter are not just capacity and speed, but the ratio between them. A Groq LPU carries only 500 MB of memory, yet moves it at 150 TB/s, which is a velocity tier, not a capacity tier. A Rubin GPU balances 288 GB against 22 TB/s. A Vera CPU trades speed for reach, holding 1.5 TB at a comparatively gentle 1.2 TB/s. Divide bandwidth by capacity and the three chips reveal their true jobs.

Table 3. The three Vera Rubin execution engines. Bandwidth per gigabyte is the tell: it explains why SRAM is a speed tier and LPDDR5X is a capacity tier. Source: NVIDIA Vera, Rubin and Groq 3 LPX materials, July 2026, preliminary.
EngineLocal memoryBandwidthBandwidth per GBBest stage of the job
Vera CPUup to 1.5 TB
LPDDR5X
1.2 TB/s~0.8 GB/sOrchestration, agents, tools, ETL, KV-cache
Rubin GPU288 GB
HBM4
22 TB/s~76 GB/sPrefill, attention, training, flexible inference
Groq 3 LPU500 MB
SRAM
150 TB/s~300,000 GB/sLatency-sensitive decode, FFN and MoE experts

This is why the Rubin GPU and the Groq LPU ship as partners rather than rivals. In a modern reasoning workload, the prefill and attention phases want the GPU's huge, fast HBM, while the token-by-token decode phase wants the LPU's tiny, blindingly fast SRAM and its deterministic schedule. NVIDIA's Dynamo software routes each stage to the engine that suits it, so a single request can be served by three different kinds of chip without the user ever knowing. For a buyer, the lesson is not "order a Groq rack." It is that the future of inference is heterogeneous, and your target operating model should be written in terms of stages and their needs, not a single box that does it all.

NVIDIA Dynamo routes each stage to the engine that fits it Request user or agent Vera CPU orchestrate, KV cache high reach, low tensor Rubin GPU prefill + attention big, fast HBM4 Groq 3 LPU token decode ultra-low latency Response streamed
Figure 5. One inference request, three kinds of silicon. NVIDIA Dynamo sends orchestration to the Vera CPU, the compute-heavy prefill and attention to the Rubin GPU, and the latency-critical decode to the Groq LPU. The precision and bandwidth choices from earlier sections decide where each stage lands. Alt text: left-to-right pipeline from request through Vera CPU, Rubin GPU and Groq LPU to response, coordinated by Dynamo.

None of this changes the discipline. If anything it sharpens it, because a disaggregated platform rewards buyers who know their workload in detail and punishes those who do not. The organisation that mapped its stages can compose exactly the right mix of Vera, Rubin, and Groq. The organisation that just wanted "the new NVIDIA thing" will overbuy on one axis and starve another. For the deeper architecture, our companion piece on the Vera Rubin NVL72 rack walks the physical build, and our essay on why NVIDIA is not really a GPU company anymore explains the software and systems moat that makes this whole platform sticky.

07/ The blueprint, step by step

A repeatable way to buy NVIDIA infrastructure without buying regret

Everything so far becomes a sequence you can run for any workload, any brand, any budget. It is deliberately unglamorous, because the glamour is what gets organisations into trouble. Follow it in order and each line of your eventual bill of materials will trace back to a use case and a payback, which is the only defence that survives a budget review.

STEP 1Currentoperating model STEP 2Targetoperating model STEP 3Constraint &precision STEP 4Platformshortlist STEP 5 · FACILITY & DATA-PATH GATE power · cooling · fabric · storage.  not ready? fix this before you buy STEP 6 · Deploy at a scalable unit, then scale on evidence
Figure 6. The decision flow. Four steps define the need, one gate decides whether your building can honour it, and only then do you deploy, starting small and scaling on evidence. The facility gate is drawn wide on purpose: it vetoes more deployments than performance ever does. Alt text: flow from current operating model to target operating model to constraint and precision to platform shortlist, through a facility and data-path gate, to a staged deployment.
  1. Map the current operating model. Write down what you actually run today: the workloads, their model sizes, their latency commitments, their utilisation, and the precision they use right now. You cannot improve a picture you have not drawn, and most organisations have never drawn this one honestly.
  2. Define the target operating model. Decide where you need to be in eighteen to thirty six months, expressed as use cases that create value, not as hardware you admire. This is the document the whole purchase serves, and it is worth more than any vendor deck.
  3. Name the binding constraint, and the precision. For each workload, say plainly whether it is limited by latency, raw compute, memory bandwidth, or working-set reach, and at which numeric format it runs. This single sentence per workload does more to size the estate than any benchmark.
  4. Shortlist platforms by fit, not by prestige. Match each workload to the smallest system that satisfies its constraint, using the logic in the fit finder. Right-size the scale-up domain: do not buy a coherent rack to run a model that shards cleanly across eight cards.
  5. Clear the facility and data-path gate early. Confirm the power, the cooling, the liquid loop, the rack density, the storage throughput, and the fabric before you sign anything. The accelerators are only as fast as the data pipeline that feeds them, and a rack you cannot power is a very expensive radiator.
  6. Deploy at a scalable unit, then grow on evidence. Start at the documented scalable unit, instrument utilisation and payback, and let real numbers, not roadmap envy, pull you to the next size. Keep the Vera Rubin roadmap as an option you exercise when the operating model asks for it, never as an obligation the calendar imposes.
The one-line test Before any purchase order, finish this sentence out loud: "We are buying this platform at this precision because our target operating model needs this workload to hit this constraint, and our facility can support it today." If you cannot complete it without hand-waving, you are not ready to buy. You are ready to keep planning, which is cheaper.
Advertisement

08/ The verdict

Buy the operating model, not the GPU

The silicon in this piece is genuinely extraordinary. A single Rubin GPU quotes fifty petaflops of low-precision inference, a GB300 NVL72 turns seventy two accelerators into one coherent domain, and a Groq LPU moves half a gigabyte of memory at a hundred and fifty terabytes a second. It would be easy, and expensive, to fall in love with any of it.

The advisor's job is to love the outcome instead. The organisations that win the next few years will not be the ones that bought the biggest number. They will be the ones that knew their work well enough to buy precisely, that named their constraint before they named a product, that treated FP4 and FP64 as different strategies rather than different rows, and that walked their building's power plan before they walked a sales floor. NVIDIA has built a catalogue deep enough to serve almost any operating model. Your advantage is not access to it. Your advantage is the discipline to take only the part your strategy actually needs, and to make every watt and every dollar earn its place.

The tools have never been more powerful. The discipline to deploy them well is still the rarest component in the rack.

09/ Questions worth asking

The questions a good architect will ask you back

Which NVIDIA AI factory should I choose for inference?

It turns on latency and working-set size. For flexible enterprise inference and mixed workloads, an RTX PRO 6000 AI factory is often enough. For large-model, low-latency inference at scale, a GB300 NVL72 gives you a single 72-GPU NVLink domain. For ultra-low-latency decode in the Vera Rubin era, a Groq 3 LPX rack pairs with Rubin GPUs so each stage of the request runs on the engine that suits it.

What is the real difference between HGX B300 and GB300 NVL72?

HGX B300 is an eight-GPU server that creates an eight-GPU NVLink scale-up domain, with wide OEM choice of CPU, memory, and cooling. GB300 NVL72 is a standardised, liquid-cooled rack that links all seventy two GPUs into one NVLink domain with coherent Grace CPU memory, at up to 142 kilowatts per rack. HGX fits a conventional server model, NVL72 is a rack-scale appliance that your facility has to be built for.

Why does precision (FP4, FP8, FP16, FP32) change the hardware decision?

Precision sets both throughput and accuracy. Low precision such as NVFP4 gives the highest inference throughput and the smallest memory footprint, FP16 and BF16 are common for training that must converge cleanly, and FP32 and FP64 matter for scientific and high-precision work. A platform that is excellent at FP4 inference can be the wrong choice for FP64 HPC, so the number format belongs in the workload definition, not in a tuning session afterwards.

Is NVL72 memory really one big unified pool?

No. NVL72 provides coherent addressability between each Grace CPU and its paired Blackwell GPUs, plus a rack-wide NVLink fabric for fast peer access across seventy two GPUs. But HBM stays physically attached to each GPU and LPDDR5X stays attached to each CPU. Unified addressability is not one uniform physical pool, so locality and topology still shape performance, and your software still has to place data with intent.