I Bought NVIDIA's $4,000 'AI Supercomputer.' Then I Sold It.

The Pitch

When NVIDIA announced the DGX Spark, a desktop box built on the GB10 chip with 128GB of unified memory, I was exactly the target audience. I run local inference for a side project that processes domain-specific documents, the kind where nuance matters and getting a detail wrong has consequences. My pipeline generates N candidate summaries in parallel, then runs each one through multiple LLM-based evaluation stages that check for completeness, fidelity to the source material, and accuracy of domain terminology. Think A/B testing but with N variants and LLM judges instead of user clicks. The kind of workload where you need big models at reasonable speeds without hemorrhaging money on cloud API calls.

128GB of unified memory meant I could load models that wouldn’t fit on any consumer GPU. The NVIDIA ecosystem meant vLLM, Docker, and the inference stack I already knew. On paper, this was the machine I’d been waiting for.

I got a Founders Edition in late October 2025 for $3,999 plus tax.

What Actually Worked

I’ll say this up front: for its original intended workload, the Spark delivered.

My pipeline started as a parallel generation architecture, produce N candidate summaries simultaneously, score them for completeness and fidelity, refine the best. Running Qwen 2.5 72B across 500-600 concurrent sequences, the Spark pushed 2,500-3,000 tok/s aggregate throughput. That’s legitimate. At high batch sizes the memory bandwidth bottleneck gets amortized across sequences, and the 128GB pool meant no VRAM anxiety. Everything fit. Everything ran. For that specific workload shape, the machine earned its price tag.

The hardware itself is beautiful. Compact, well-built, runs cool and quiet under sustained load. NVIDIA’s metal-foam cooling solution is genuinely impressive engineering. If the story ended here, this would be a positive review.

It doesn’t end here.

The Quality Ceiling

My project has a quality bar that keeps going up. The documents I’m summarizing are dense, domain-specific, and full of subtle distinctions that smaller models flatten or miss entirely. The 72B dense models I started with were good enough for the initial architecture, but as requirements tightened, they weren’t capturing the nuance I needed. Summaries were technically correct but lossy, dropping qualifications, collapsing distinctions, missing implications that a domain expert would catch. So I reached for bigger models.

I moved to Qwen 3.5 122B, a MoE model with 10B active parameters. It fit in 128GB. Then I tried Qwen 3 235B (22B active) at dynamic Q2 quantization. That fit too. The Spark’s memory capacity is genuinely its superpower, you can load models that would be impossible on any consumer GPU.

But loading weights is not the same as moving them.

The Architecture Shift

As I chased quality up the model ladder, two things happened. First, no local model, not even the 235B, could generate summaries at the fidelity level I needed for the most demanding source documents. I tested generation on every frontier cloud API too, GPT-5.1, 5.2, and 5.4, Gemini 3 Flash and Pro, Claude Opus 4.6, Sonnet 4.6. OpenAI ended up producing the best results for generation at 2-5 cents per run. That part was cheap and solved.

But it wasn’t for lack of trying locally. Over five months I threw everything I could find at this machine, reasoning and non-reasoning models, dense and MoE, small and large: Qwen 2.5 32B and 72B, Gemma 3 27B, GLM 4.5 Air, GLM 4.7 Flash, MiniMax M2.5, Qwen3 Coder 30B-A3B, Qwen3 Coder Next, Qwen 3 235B-A22B, Qwen 3.5 35B-A3B, Qwen 3.5 122B-A10B, Llama 3 70B, Llama 3.1 70B, Llama 3.3 70B, Mixtral 8x7B, Mixtral 8x22B, Phi-4, Yi 1.5 34B. I even tried Nemotron, NVIDIA’s own model, which refused to run on NVIDIA’s own hardware due to compatibility issues. Let that one sit for a moment.

When local models couldn’t hit my quality bar, I rented cloud GPUs to test even bigger ones. 8xH200 and 8xB100 clusters running Llama 3.1 405B and Mistral Large 3 675B. Even those weren’t enough for generation, the quality ceiling on my specific task is brutally high. That’s what ultimately pushed generation to frontier cloud APIs.

What stayed local was the evaluation layer, and not by default. I tried offloading evaluation to every major cloud API: GPT-5.1, 5.2, and 5.4, Gemini 3 Flash and Pro, Claude Opus 4.6, Sonnet 4.6, and Haiku 4.5. None of them matched the Qwen 3.5 122B with extended reasoning on my specific evaluation criteria. My scoring pipeline needs judges that compare each candidate summary against the source document, checking for dropped details, distorted meaning, hallucinated content, and domain-specific terminology accuracy. The frontier APIs were good. The 122B MoE with deep chain-of-thought was better, more thorough, more precise, fewer false positives.

There’s also a hard architectural constraint: the evaluation model must be a different model from the one that generated the summary. This isn’t a preference, it’s a requirement. Models have systematic biases in how they produce and evaluate text. If the same model generates and judges, it tends to rate its own output patterns favorably, the same stylistic choices, the same structural habits, the same failure modes become invisible to the judge because they’re baked into its own training distribution. You get high scores that don’t reflect actual quality. Using a different model family for evaluation breaks that self-reinforcing loop and produces honest assessments. This means I can’t just use the same cloud API for both generation and scoring, and the best scorer I found happens to be a 122B open-source model, not a cloud API.

And at my volume, even the “cheap” cloud evaluation APIs added up to dollars per completed summary across multiple refinement iterations. So the evaluation layer stayed local not because I was being stubborn about cloud costs. It stayed local because the local model was the best tool for the job, it was architecturally necessary to use a different model, and the economics of cloud evaluation at scale are punishing. That’s the kind of problem that should make a device like the Spark invaluable.

But “better” came with a brutal catch.

The evaluation models need extended reasoning, deep chain-of-thought analysis where the model cross-references the candidate summary against the source document, verifying that nothing was dropped, distorted, or fabricated. For a ~2,500 token input, a single evaluation call would generate roughly 10,000 thinking tokens and a 600-token response. That’s ~13,000 tokens of output per call.

At 10 tok/s on the Spark.

That’s roughly 20 minutes for a single scoring call. I have multiple evaluation stages. Even batched, the full pipeline wall time was around 45 minutes.

The economics told the same story from a different angle. Generating candidate summaries costs pennies per run on cloud APIs, not worth optimizing. Evaluation costs 40-60 cents per iteration on cloud inference providers, and my pipeline runs up to five refinement iterations per completed summary. Evaluation is 10-15x more expensive than generation. At my volume, those costs add up fast.

So the machine that was supposed to eliminate cloud costs became the bottleneck that made my pipeline unusable. Not because it lacked the memory to load the models. Because it couldn’t move the weights fast enough.

The Number NVIDIA Didn’t Want You to See

Here’s where things get pointed.

The DGX Spark’s memory bandwidth is 273 GB/s.

You won’t find that number in the marketing. NVIDIA led with “128GB unified memory.” They led with “1 PFLOP of FP4 compute.” They led with “runs models up to 200 billion parameters.” All technically true. All carefully chosen to obscure the one number that actually determines whether this machine is usable for inference: how fast can you read from memory?

LLM inference, particularly the autoregressive token generation that dominates real workloads, is almost entirely memory-bandwidth bound. For dense models, you’re reading the full model weights for every single token. Nobody runs 72B at FP16 in practice, you’d quantize to Q8 (~72GB) or Q4 (~36GB) and accept the precision tradeoff. But even at Q4, a 72B dense model needs to stream ~36GB of weights through memory per token. At 273 GB/s, that gives you roughly 7-8 tok/s at best for single-request inference. Usable for a chatbot. Not usable for a pipeline that needs hundreds of calls.

MoE architectures help, a 122B model with 10B active parameters only needs to read the active expert weights per token, not the full 122B. So despite all the weights being resident in memory, you’re moving a fraction of them per inference step. This is why MoE models are the sweet spot for bandwidth-constrained hardware, and why I gravitated toward them. But “a fraction” of a very large number is still a meaningful number, and at 273 GB/s even the MoE advantage only gets you so far.

At 273 GB/s, large models crawl. The “up to 200B models” claim works like this: a 200B model at INT4 is roughly 100GB, leaving 28GB for KV cache. The weights load. The model runs. At speeds that make it a paperweight for anything interactive.

It’s like advertising an EV with 5,000-mile range and neglecting to mention the top speed is 5 mph.

For comparison, the M3 Ultra Mac Studio, the machine I eventually bought, runs at 819 GB/s. Three times the bandwidth. A consumer RTX 5090 hits 1,792 GB/s, over 6x the Spark. An H200 datacenter GPU does 4,800 GB/s. A B200 does 8,000 GB/s, nearly 30x what the Spark delivers. The DGX Spark, marketed as an “AI supercomputer,” has less memory bandwidth than a MacBook chip and roughly one-sixth of the consumer GPU that NVIDIA sells to gamers.

That’s not a marginal shortfall. That’s a product positioned at the absolute bottom of the inference performance ladder while being marketed as if it belongs near the top. The 128GB memory pool is the only dimension where the Spark wins, and NVIDIA leaned into that number precisely because it’s the only one that looks good.

NVIDIA knows this. They chose not to advertise it. That wasn’t an oversight.

My Part in This

I need to own something here. I fell for it.

When I pre-ordered the Spark, I didn’t understand how LLM inference actually works at the silicon level. GPU memory architecture wasn’t my world. For the inference work, I operate above the GPU, Python, vLLM, Docker, API orchestration. I knew how to serve models, how to batch requests, how to build evaluation pipelines. I did not know that autoregressive token generation is almost entirely memory-bandwidth bound, or what that means in practice for hardware selection.

I saw 128GB of unified memory, I saw “Blackwell,” I saw “1 PFLOP,” and I trusted NVIDIA, a company I’d relied on for decades, to ship a product that actually worked for the use case they were marketing it for. And that’s exactly the profile NVIDIA was targeting: application developers who want local inference without needing a PhD in GPU microarchitecture. People who trust the spec sheet because they don’t have the background to question it.

That’s on me. But it’s also precisely the point. If the target audience for this product needs to independently derive the relationship between memory bandwidth and token generation speed to avoid a $4,000 mistake, then NVIDIA has either built the wrong product or marketed it to the wrong people. I suspect they know exactly what they did.

The LPDDR5X Question

Why 273 GB/s? The GB10 uses LPDDR5X memory. HBM3e would have been impractical at this price point and thermal envelope, fair enough. But GDDR6 or GDDR7? The RTX 5090 ships with GDDR7 at 1,792 GB/s in a consumer card. Even GDDR6 would have delivered a meaningful bandwidth improvement. And here’s the thing, even within LPDDR5X, the GB10 is leaving bandwidth on the table. Apple’s M3 Ultra uses the same memory technology and achieves 819 GB/s by running a wider memory bus. Same type of chip, 3x the bandwidth, because Apple invested in the bus width. NVIDIA chose both the lowest-bandwidth memory type available and a narrow bus configuration within that type. That wasn’t purely about cost or thermals. That was a product decision.

I think the answer is straightforward: if the DGX Spark had datacenter-class memory bandwidth, it would cannibalize NVIDIA’s own cloud GPU business. The entire revenue model of selling H100s and B200s to cloud providers depends on local inference hardware being just good enough to keep developers in the CUDA ecosystem but not good enough to replace cloud API calls for production workloads.

The Spark is a $4,000 ecosystem lock-in device. It keeps you writing CUDA, buying into NVIDIA’s stack, and ultimately renting datacenter GPUs when you need real performance. The 128GB of memory is the bait. The 273 GB/s bandwidth is the leash.

The Software Catastrophe

If the bandwidth story was the strategic kneecapping, the software story is the operational abandonment.

“Blackwell” Means Two Different Things

NVIDIA calls the GB10 a “Grace Blackwell Superchip.” What they don’t clarify is that the GB10’s compute architecture (SM121, compute capability 12.1) is fundamentally different from datacenter Blackwell (SM100). The major version numbers themselves diverge,10.x for datacenter, 12.x for consumer/edge, and 11.x is entirely vacant. No prior NVIDIA generation has produced this wide an architectural gap under a single brand name.

The datacenter Blackwell chips have TMEM (dedicated tensor memory), the tcgen05 instruction, and WGMMA (warp group matrix multiply accumulate). The GB10 has none of these. Those instructions were replaced with RT cores and the older mma.sync approach. The GB10 can do FP4 math through its 5th-gen tensor cores, but every software implementation in the ecosystem, FlashInfer, FlashAttention, FlashMLA, CUTLASS, was written against SM100’s instruction set.

So when you hear “Blackwell” and expect software compatibility, you’re in for a surprise. The kernels don’t work. They need to be rewritten from the ground up for SM121.

NVFP4: The Flagship Feature That Didn’t Exist

When Jensen Huang unveiled Project DIGITS at CES 2025, the headline number was “1 petaflop of AI performance at FP4 precision.” It was the biggest number on the spec sheet, the one that made the product sound like a supercomputer. NVFP4 is NVIDIA’s proprietary 4-bit floating-point format with hardware-accelerated tensor core datapaths, their pitch was that it preserves more precision than generic INT4 quantization while running faster through dedicated silicon. The “1 PFLOP” claim only exists at FP4 precision.

Here’s the thing: 4-bit quantization already existed. The community was already running Q4 and INT4 quants on everything from RTX cards to Apple Silicon. NVFP4’s value proposition was the hardware acceleration, dedicated FP4 datapaths in the tensor cores that would be faster than software-level quantization. And on a bandwidth-constrained device like the Spark, that acceleration matters even more, because 4-bit means half the data movement of 8-bit, which is exactly what you need when your memory bus is narrow. A wider bus would have solved the problem directly. Instead, NVIDIA chose the narrow bus and bet on a proprietary quantization format to paper over it.

On the GB10, NVFP4 was broken from day one. The CUTLASS FP4 GEMM kernels crashed because the SM120 tile configurations require more shared memory than the GB10 provides (99 KiB vs the B200’s ~228 KiB). FlashInfer detected the GB10 wasn’t SM100, fell back to CUTLASS, and CUTLASS failed too. The hardware supports FP4 math at the tensor core level, but the entire software stack to use it didn’t exist.

It took until late February 2026, four months after the Spark shipped, for a community contributor to finally get working NVFP4, “after months of iteration and research.” Not NVIDIA. A random person on the developer forums.

vLLM: Two Versions Behind, Always

Instead of upstreaming GB10 support to mainline vLLM, NVIDIA maintains a walled-garden Docker fork that consistently lags two major versions behind. When I was running the Spark, their latest image was on vLLM 0.12-0.13 while mainline was at 0.15+. The February 2026 image finally shipped 0.15.1, but mainline had already moved to 0.17.1.

This matters because when new models drop, and they drop fast, you need latest vLLM. Model developers like Alibaba’s Qwen team integrate with mainline and ensure compatibility before release. NVIDIA’s fork doesn’t get those updates for weeks or months. So when Qwen 3.5 launched, mainline vLLM was ready on day one. The Spark’s official image? You’re waiting.

The community response tells you everything. People on the NVIDIA developer forums are asking “can anyone suggest a tag of vLLM that works reliably on the DGX Spark?” One user titled their thread “VLLM, the $150M train wreck?” Multiple independent contributors maintain their own Docker build scripts, patch sets, and nightly images, explicitly noted as “not affiliated with NVIDIA.” As of March 2026, five months post-launch, vLLM still does not officially support the DGX Spark platform.

The Missing Config File

Here’s a detail that captures the entire software story in miniature. vLLM ships pre-tuned configuration files for known GPU architectures, JSON files that optimize MoE kernel dispatch parameters for specific hardware. They’re generated by running benchmarks on the target GPU. NVIDIA ships these configs for their datacenter GPUs.

They did not ship one for the GB10.

So when you run a MoE model on the Spark, vLLM hits: Config file not found at .../configs/E=128,N=1856,device_name=NVIDIA_GB10.json and falls back to untuned defaults. Community members who generated these configs and mounted them as Docker volumes reported up to 2x speedups.

A JSON file. Generated by running a benchmark. Worth up to double the performance. NVIDIA couldn’t be bothered to create it before shipping a $4,000 machine. And as of March 2026, five months after launch, they still haven’t. The community generates and shares these configs among themselves. NVIDIA apparently has other priorities.

A $4,000 Dev Board in a Shiny Box

Let me be precise about what the DGX Spark actually is. It’s a developer kit. A reference platform for the GB10 silicon. The kind of thing NVIDIA would normally sell to OEMs and partners for integration testing, packaged in a premium enclosure and marketed as a “personal AI supercomputer” to capture the local-inference gold rush.

NVIDIA did zero work to make sure the people paying $4,000 (now $4,699 after a February 2026 price hike driven by memory supply constraints) would have a functional software stack. The NVFP4 that’s on the box doesn’t work. The vLLM they ship is stale. The tuning configs don’t exist. The architecture compatibility is a lie of omission. And when things break, the response is a Docker image update two months later with a version that’s still behind mainline.

If you’re the kind of person who enjoys being a beta tester, who wants to contribute kernel patches upstream and maintain custom build scripts, the Spark is a fascinating piece of hardware. That’s not a sarcastic caveat. The community around this thing is impressive and the engineering challenges are genuinely interesting.

But I didn’t buy a project. I bought a tool. And the tool didn’t work.

Where the Spark Goes From Here

To be fair to the hardware: the Spark’s story isn’t over. The community is making real progress. The MoE tuning configs are getting generated and shared. The NVFP4 workarounds are maturing. vLLM support for SM121 is improving with every community build. The GatedDeltaNet fixes, the FlashInfer patches, the CUTLASS tile configurations, all of this is getting better month over month. And the new wave of MoE models from Qwen and others are practically designed for hardware like the Spark: large total parameter counts with small active parameters, which partially mitigates the bandwidth constraint by reducing how much weight data needs to move per token.

If you bought a Spark today and ran Qwen 3.5 35B-A3B or Qwen3 Coder 30B-A3B with all the community fixes applied, the MoE tuning configs mounted, and the latest patched vLLM build, you’d have a meaningfully better experience than I had five months ago. The machine would feel closer to what NVIDIA promised.

But it has a ceiling, and the ceiling is 273 GB/s. No amount of software optimization changes the memory bus. The Spark will never be faster than a consumer gaming GPU at single-request inference, an RTX 5090 at 1,792 GB/s will smoke it on raw token generation speed for any model that fits in 32GB of VRAM.

The Spark’s advantage is and always will be the 128GB memory pool. It can load models that no consumer card can touch. But for models that do fit on a gaming GPU, the gaming GPU wins on speed every time. And if you need both the memory and the speed? Your options are: stack multiple consumer GPUs with tensor parallelism across PCIe (which adds complexity and hits the PCIe bandwidth wall), hunt for older NVLink-capable cards (good luck), or buy a Mac with unified memory and 3-6x the bandwidth.

The Spark carved out a real niche, big models, modest speeds, CUDA ecosystem. For the right workload at the right expectations, it earns its keep. It just wasn’t my workload anymore.

The Decision

I sold the DGX Spark in early March 2026 for $3,800, a $200 haircut plus tax on five months of ownership. Given that NVIDIA raised the price to $4,699 two weeks earlier, the buyer got a deal. I got out clean.

I ordered an M3 Ultra Mac Studio. Base config (28-core CPU, 60-core GPU), upgraded to 256GB unified memory. 819 GB/s bandwidth. The MLX inference ecosystem. No CUDA compatibility matrix. No FlashInfer kernel issues. No hybrid architecture surprises.

I would have ordered the 512GB configuration. Apple pulled it from the lineup approximately a week before I placed my order, another casualty of the global DRAM shortage that also drove the Spark’s price increase. The 256GB upgrade cost me $2,000, up from $1,600 a week earlier. Same memory crunch, different product, same story.

The Mac Studio arrives in May 2026. In the meantime, my RTX 4060 Ti machine handles what it can, and cloud APIs fill the gaps.

Did I consider waiting for the M5 Ultra Mac Studio expected later this year? Yes. The M5 Max runs at 614 GB/s, and if Apple follows the UltraFusion pattern of doubling the Max die, an M5 Ultra could theoretically hit ~1.2 TB/s, a 50% jump over the M3 Ultra’s 819 GB/s. But that’s all speculation. No M5 Ultra has been announced. No specs have been confirmed. And five months of 45-minute pipeline runs taught me the value of a bird in the hand. The M3 Ultra at 3x the Spark’s bandwidth is real, it’s ordered, and it arrives in May.

The Bigger Picture: Who’s Actually Building the Future

Here’s where this stops being a product review and starts being a thesis.

The US Playbook

The dominant strategy from US AI companies has been: invest billions in training, gatekeep the resulting models behind API paywalls, open source only the small models that don’t threaten the revenue moat, and build the entire value chain around cloud inference. NVIDIA sells the datacenter GPUs. The labs sell API access. Cloud providers take a cut. Everyone makes money. Nobody runs anything locally. Everyone pays rent.

The “open source” releases from US labs have been, with few exceptions, carefully sized to be impressive demos but inadequate for serious production workloads. Small enough to generate goodwill and developer adoption. Not big enough to cannibalize API revenue.

The Chinese Counterstrategy

Then the Chinese labs showed up with a fundamentally different approach.

They can’t outspend the US labs on compute. They can’t even buy the latest NVIDIA GPUs, the export ban is supposed to prevent exactly this kind of competition. Some of them are training on Huawei Ascend clusters because that’s what they have access to.

So they innovated under constraint. And anyone who’s ever built anything knows what happens when you add constraints to a creative process: you get better solutions. The constraint doesn’t kill the will, it redirects it. The same dynamic that makes the demoscene produce better code in 64KB than most developers write with unlimited memory. The same reason every brainstorming technique worth a damn involves adding artificial limitations to force creative thinking.

The result has been devastating.

DeepSeek’s training breakthroughs in MoE efficiency. Qwen filling every gap in the model size range, from 0.8B to 397B, dense and MoE, with the 100B-200B sweet spot that makes local inference genuinely viable. Kimi K2.5 at one trillion parameters with open weights. GLM-5 trained entirely on domestic Chinese hardware and still competing with frontier Western models. All open-weight. All priced at fractions of US API costs, DeepSeek’s API runs at 140x cheaper than comparable US offerings.

Qwen has overtaken Meta’s Llama in cumulative downloads on Hugging Face. Each time a Chinese model drops, it compresses the competitive position of every US lab charging API rent. Qwen 3.5, Kimi K2.5, MiniMax M2.5, GLM-5, they keep coming, they keep getting better, and they keep being open.

The Model Size Gap Was the Moat

Until the Chinese labs filled it, there was a convenient dead zone in the open model landscape. Models were either small enough to run on consumer GPUs (up to ~70B) or too large for anything short of multi-GPU datacenter setups (400B+). The 100B-200B range, where a device like the Spark or Mac Studio could be genuinely useful for serious local inference, was largely empty.

That gap was the cloud providers’ moat. Small models don’t need expensive hardware. Giant models can’t run locally. So you rent.

The Chinese labs filled the moat. Qwen 3.5 at 397B total / 17B active. Qwen 3.5 122B at 10B active. Models specifically architected for MoE efficiency at the scale that maxes out a 128-256GB local machine. These are the models that make local inference competitive with cloud APIs for real workloads, and in my case, the local model is better than the cloud alternative for the task that matters most.

What I Learned

The DGX Spark taught me three things:

Bandwidth is the number that matters most, once you have sufficient compute. Any modern GPU has enough TOPS to handle inference. The bottleneck is almost never compute, it’s how fast you can feed the weights to the compute units. TOPS without bandwidth is a engine without fuel. Bandwidth without TOPS is a fire hose pointed at a thimble. But in practice, every piece of inference hardware shipping today has more than enough compute. What separates usable from unusable is bandwidth. 273 GB/s vs 819 GB/s. That’s the story.

Follow the incentives. NVIDIA needs you renting datacenter GPUs. The Spark’s bandwidth was chosen to keep you dependent on cloud inference for production workloads. The software stack was neglected because supporting local inference isn’t where the money is. When a company’s product strategy conflicts with your use case, the product will always serve the strategy.

The future of local AI inference is being built in Hangzhou, not Santa Clara. Apple builds the hardware with the bandwidth. Chinese labs build the models with the efficiency. The winning local inference stack in 2026 is Apple Silicon running Chinese open-source models through MLX, no CUDA, no API rent, no permission needed. The two largest US monopolies in AI, NVIDIA’s hardware dominance and the frontier labs’ API gatekeeping, are being outflanked simultaneously from opposite directions by forces neither of them controls.

NVIDIA will probably get the Spark’s successor right. Faster memory, working software, proper kernel support. And by then, the M5 Ultra will be shipping at 2x the bandwidth, Qwen 4 will be running locally on it, and the window for NVIDIA to own the desktop inference market will have closed.

The DGX Spark should have just worked. I don’t have more time to waste. I have shit to do.

Opinions are the author’s own. The DGX Spark was purchased at retail and sold at a loss. No vendor provided hardware, access, or compensation for this review.