Local AI box vs cloud: should you buy your agent a computer?
Local AI hardware is really a memory purchase, and memory prices surged in 2026, with vendors saying so openly. The ladder from $249 to $6,950, the math against subscriptions, and when local wins.
Buying a local AI box is really buying memory. Model size is capped by RAM, not clock speed, and 2026 is a rough year to buy memory: DRAM contract prices nearly doubled in a single quarter, and NVIDIA raised its desktop AI box’s price citing exactly that. Local wins when privacy is absolute, when you genuinely run models all day, or when tinkering is the point. Renting wins on math for almost everyone else: a $2,000 box bought to avoid a $20–30 monthly bill takes years to pay off and never gets frontier-model quality. Most practitioners land on a split: cheap local for the private and simple, cloud for the hard.
You’re buying memory, not speed
The sizing rule is unglamorous: a model has to fit in memory before speed matters at all. Ollama’s long-standing guidance (since removed from its README, but still the right mental model) put it as 8GB of RAM for 7B models, 16GB for 13B, 32GB for 33B. The vendors now say the quiet part themselves. AMD’s own AI blog: “the amount of memory available for AI accelerators has become a key bottleneck.” That’s why the interesting local machines are unified-memory designs: AMD’s 128GB Ryzen AI Max+ can run 4-bit models up to ~128B parameters, and NVIDIA rates the 128GB DGX Spark for inference up to ~200B. Meanwhile a gaming GPU’s 24GB, whatever its horsepower, caps you far lower. Memory is the product.
Which is why 2026 pricing hurts
TrendForce measured conventional DRAM contract prices up 93–98% quarter-over-quarter in early 2026, with another ~60% rise forecast for the next quarter. The knock-ons are visible on vendor pages, not just in analyst notes: NVIDIA raised the DGX Spark from $3,999 to $4,699 in February 2026 and said why in as many words: “the price adjustment reflects industry wide memory supply constraints.” NVIDIA’s own store lists it at $6,950 today, out of stock. Framework’s 128GB Strix Halo desktop launched at $1,999 in 2025 and lists at $3,449 today, and Framework says the prices of the memory chips that configuration uses “have spiked the most.” The big-memory rungs of the local ladder got more expensive to climb, for the same underlying reason: you’re bidding for RAM against the entire AI industry.
The ladder, priced (as of October 2026)
- The laptop you own: $0. With 16GB it runs 7–13B models via Ollama + Open WebUI. The right first rung: it tells you whether you actually use local models before you spend anything.
- Jetson Orin Nano Super: $249 (NVIDIA’s price in August 2026). NVIDIA now sells it only through partners. 8GB, so small-model territory: a tinkerer’s box, not a daily driver.
- Mac mini M6: $899 (16GB) / $1,099 (24GB). Quiet, 4W idle / 70W max, and the most livable general-purpose rung; 13B-class models comfortably.
- Used RTX 3090: commonly ~$800 (August 2026). The CUDA path: 24GB VRAM, big-GPU speed, gamer-PC economics. Plus the noise, wattage, and used-market roulette.
- Framework Desktop (Ryzen AI Max+): $1,959 (64GB) / $3,449 (128GB). The unified-memory sweet spot for genuinely large local models, when it’s in stock.
- DGX Spark: $6,950. 128GB, ~200B parameter ceiling, a petaflop on your desk. The no-compromise local rung, at a price that says so.
The math against the subscription
Do the arithmetic before the vibes: $2,000 of hardware against a $20–30/month cloud spend is a five-to-eight year payback, longer than the hardware’s useful life in a field moving this fast, and it assumes the local model actually substitutes for what you were paying for. Often it doesn’t: local 13–30B models are genuinely useful, but the hard reasoning still goes to frontier models, which is why the end-state for most people is a split, not a conversion. Electricity, for fairness, is the one cost that’s smaller than feared: a Mac mini idles at 4 watts.
When local genuinely wins
Three cases hold up. Absolute privacy: some data should never leave the building, full stop. No cloud policy substitutes for physics. Heavy sustained use: if you’re running models many hours a day, the payback math flips. And the third: it’s a hobby that pays dividends, the same instinct behind agents built to run local models and self-hosted everything. If that’s you, buy the box; you were always going to.
The other side of the ledger
Renting an agent a computer isn’t renting a GPU. That’s the category confusion this decision usually hides. An always-on agent needs a machine (persistence, uptime, its memory and files somewhere safe) far more than it needs local inference: model thinking arrives by API or an existing subscription either way. That’s the shape of a managed always-on pod at $29/mo: no memory-market exposure, no 140-watt space heater, and the frontier models your hardest questions deserve. Where should the agent itself live? That question has its own comparison; this one is simpler than it looked: unless privacy, volume, or joy says otherwise, keep your capital and rent the computer.