# What’s the Best AI Model Your GPU Can Actually Run?16GB to 1TB VRam
Canonical: https://social-archive.org/jack_bauer111/Dz3GfRV7l5
Original URL: https://www.youtube.com/watch?v=kTmUbRVsamo
Author: Kai
Platform: youtube
Share mode: full
## Content
What AI can you actually run locally at every memory size, from 16GB to 1TB? I walk the whole local AI ladder (16GB, 32GB, 64GB, 128GB, 256GB, 512GB and 1TB) and show the best local LLM for each rung, what it scores, what the hardware costs, and which doublings buy a smarter model versus the same model with more bits. Most "how much RAM for local AI" tier charts draw a smooth line. On the Artificial Analysis Intelligence Index it's a staircase: Qwen 3.8 27B scores 20.1 at 16, 32 and 64GB. GLM-5.3-Flash brings 27.8 at 128GB. DeepSeek V4.1 Flash brings 32.5 at 256GB, and the same 32.5 at 512GB with its fourth bit back. Then Kimi K3.1 Pro (46.0, weights promised for November 4) turns the 1TB rung into the biggest jump on the ladder. The best local AI model for each memory size: ▸ 16GB: Qwen 3.8 27B at 3-bit on an RTX 5060 Ti 16GB. On a 16GB Mac mini, Gemma 4 12B ▸ 32GB: Qwen 3.8 27B at 4-bit with a 128K context ($1,299 Mac mini) ▸ 64GB: Qwen 3.8 27B at 8-bit with the full 256K context, plus room for multiple agents (M5 Pro Mac mini, $2,699) ▸ 128GB: GLM-5.3-Flash or Qwen 3.8 Flash-Next (Mac Studio M5 Max, NVIDIA DGX Spark, AMD Strix Halo) ▸ 256GB: DeepSeek V4.1 Flash at 3-bit (Mac Studio M5 Ultra, or two DGX Sparks) ▸ 512GB: DeepSeek V4.1 Flash at 4-bit. Two 256GB Mac Studios cost less than one 512GB ▸ 1TB: Kimi K3.1 Pro at 2-bit (711–861GB builds) Also covered: the macOS GPU memory limit (one terminal command decides whether a Mac Studio runs the bigger model), DGX Spark vs Mac Studio prompt speed, running big MoE models on a PC with GPU + RAM + SSD (Strata), why renting a model runs the opposite way to buying hardware (total vs active parameters), and the speed ceiling at 1TB. ⏱️ Chapters 0:00 What AI you can run at every memory size 1:49 Why "more RAM = smarter AI" charts mislead 4:03 16GB: Qwen 3.8 27B vs Gemma 4 12B 5:42 32GB: Qwen 3.8 27B with 128K context 6:34 64GB: 8-bit, 256K context and multiple agents 8:24 128GB: Mac Studio M5 Max vs DGX Spark vs Strix Halo 11:07 256GB: DeepSeek V4.1 Flash on the M5 Ultra 13:38 512GB Mac Studio ($22,299) vs two 256GB 15:53 1TB: Kimi K3.1 Pro, the biggest jump 18:53 GPU + RAM + SSD: running big models on a PC 20:45 Renting vs owning: why the bill runs backwards 23:24 The speed ceiling nobody advertises 25:08 Who actually needs 1TB 26:29 The whole ladder in one minute 28:23 Rent first, then buy the smallest rung that fits 📺 Watch next Every Way to Get 128GB VRAM for Local AI → https://youtu.be/NA9D49kQFuU Every Way to Get 256GB of VRAM for Local AI → https://youtu.be/JPHT9AvoPSA Every Way to Get 512GB of VRAM for Local AI → https://youtu.be/6tihP50zlcE Every Way to Get 32GB VRAM for Local AI → https://youtu.be/uavLRbbfM94 Is 16GB All You Need for Serious Local LLM? → https://youtu.be/i9mRnbdDYxk What It Actually Costs to Run DeepSeek V4.1 Flash Locally → https://youtu.be/04ePeOyRN0k Run 125B Models 6× Faster Than llama.cpp (Strata) → https://youtu.be/m0VHx73SAG0 Why Your Mac Is Slow at Local AI (And It's Not the Chip) → https://youtu.be/x_qmqqp6d7s Can a Mac Mini Replace Your $200 AI Subscription? → https://youtu.be/3R7yba577uA 📊 Sources Artificial Analysis Intelligence Index v5: https://artificialanalysis.ai/leaderboards/models Unsloth quant sizes: https://unsloth.ai/docs/models/kimi-k3 OpenRouter model pricing: https://openrouter.ai Prices and specs as of late October / November 1, 2026. Kimi K3.1 Pro's weights are promised, not yet released, and no one has scored the 2-bit build. #LocalAI #LocalLLM #MacStudio --- ## Transcript [0:00](https://www.youtube.com/watch?v=kTmUbRVsamo&t=0s) So, I did something stupid last week. I [0:02](https://www.youtube.com/watch?v=kTmUbRVsamo&t=2s) watched five of those what AI can you [0:05](https://www.youtube.com/watch?v=kTmUbRVsamo&t=5s) ram run tier videos on YouTube and [0:07](https://www.youtube.com/watch?v=kTmUbRVsamo&t=7s) walked away really confident that I had [0:09](https://www.youtube.com/watch?v=kTmUbRVsamo&t=9s) the whole thing figured out. I bought a [0:11](https://www.youtube.com/watch?v=kTmUbRVsamo&t=11s) 64 GB machine for around $2,700, [0:15](https://www.youtube.com/watch?v=kTmUbRVsamo&t=15s) set it up on my desk like it was some [0:17](https://www.youtube.com/watch?v=kTmUbRVsamo&t=17s) kind of holy artifact, and ran my first [0:19](https://www.youtube.com/watch?v=kTmUbRVsamo&t=19s) model on [music] it. And it ran the [0:21](https://www.youtube.com/watch?v=kTmUbRVsamo&t=21s) exact same model as a $1,300 [0:24](https://www.youtube.com/watch?v=kTmUbRVsamo&t=24s) machine. It wasn't a slightly worse [0:26](https://www.youtube.com/watch?v=kTmUbRVsamo&t=26s) version, [0:27](https://www.youtube.com/watch?v=kTmUbRVsamo&t=27s) >> [music] [0:27](https://www.youtube.com/watch?v=kTmUbRVsamo&t=27s) >> and it wasn't the same model with half [0:29](https://www.youtube.com/watch?v=kTmUbRVsamo&t=29s) the features. It was the same model with [0:31](https://www.youtube.com/watch?v=kTmUbRVsamo&t=31s) the same intelligence score and the same [0:33](https://www.youtube.com/watch?v=kTmUbRVsamo&t=33s) answers, and the only difference was [0:35](https://www.youtube.com/watch?v=kTmUbRVsamo&t=35s) that my setup had the fourth bit back. [0:37](https://www.youtube.com/watch?v=kTmUbRVsamo&t=37s) [music] [0:37](https://www.youtube.com/watch?v=kTmUbRVsamo&t=37s) That sounds meaningful until you realize [0:40](https://www.youtube.com/watch?v=kTmUbRVsamo&t=40s) most people watching tier videos have [0:41](https://www.youtube.com/watch?v=kTmUbRVsamo&t=41s) never once in their life thought about [0:43](https://www.youtube.com/watch?v=kTmUbRVsamo&t=43s) what a [music] fourth bit means. And [0:46](https://www.youtube.com/watch?v=kTmUbRVsamo&t=46s) that was $1,400 [0:48](https://www.youtube.com/watch?v=kTmUbRVsamo&t=48s) extra I spent on bits. And every tier [0:50](https://www.youtube.com/watch?v=kTmUbRVsamo&t=50s) video I watched was technically correct. [0:53](https://www.youtube.com/watch?v=kTmUbRVsamo&t=53s) They just left out the part where most [0:54](https://www.youtube.com/watch?v=kTmUbRVsamo&t=54s) of the doublings on the memory ladder [0:56](https://www.youtube.com/watch?v=kTmUbRVsamo&t=56s) don't actually buy you a smarter AI. [0:58](https://www.youtube.com/watch?v=kTmUbRVsamo&t=58s) They buy you the same AI, better [1:00](https://www.youtube.com/watch?v=kTmUbRVsamo&t=60s) preserved and with more breeding room, [1:02](https://www.youtube.com/watch?v=kTmUbRVsamo&t=62s) which sounds nice until you realize you [1:04](https://www.youtube.com/watch?v=kTmUbRVsamo&t=64s) were shopping for brains [music] and not [1:06](https://www.youtube.com/watch?v=kTmUbRVsamo&t=66s) a better storage unit. So, today I'm [1:08](https://www.youtube.com/watch?v=kTmUbRVsamo&t=68s) going to walk you through the entire [1:09](https://www.youtube.com/watch?v=kTmUbRVsamo&t=69s) memory ladder, from 16 GB all the way up [1:12](https://www.youtube.com/watch?v=kTmUbRVsamo&t=72s) to a full terabyte. [music] [1:14](https://www.youtube.com/watch?v=kTmUbRVsamo&t=74s) And I'm going to tell you the real [1:15](https://www.youtube.com/watch?v=kTmUbRVsamo&t=75s) story, which isn't the smooth, clean, [1:18](https://www.youtube.com/watch?v=kTmUbRVsamo&t=78s) more RAM equals smarter AI curve the [1:20](https://www.youtube.com/watch?v=kTmUbRVsamo&t=80s) other videos draw. [1:22](https://www.youtube.com/watch?v=kTmUbRVsamo&t=82s) It's the messy one, with the doublings [1:23](https://www.youtube.com/watch?v=kTmUbRVsamo&t=83s) that bought nothing, and the one monster [1:26](https://www.youtube.com/watch?v=kTmUbRVsamo&t=86s) leap at the top that almost nobody has [1:28](https://www.youtube.com/watch?v=kTmUbRVsamo&t=88s) talked about yet. [1:29](https://www.youtube.com/watch?v=kTmUbRVsamo&t=89s) >> [music] [1:29](https://www.youtube.com/watch?v=kTmUbRVsamo&t=89s) >> And that leap completely changes the [1:32](https://www.youtube.com/watch?v=kTmUbRVsamo&t=92s) answer. Because as of right now, and I [1:34](https://www.youtube.com/watch?v=kTmUbRVsamo&t=94s) mean this week, the terabyte tier went [1:36](https://www.youtube.com/watch?v=kTmUbRVsamo&t=96s) from the rung everyone called a waste of [1:39](https://www.youtube.com/watch?v=kTmUbRVsamo&t=99s) money to the rung that holds the best [1:41](https://www.youtube.com/watch?v=kTmUbRVsamo&t=101s) open AI model in the world. [1:43](https://www.youtube.com/watch?v=kTmUbRVsamo&t=103s) >> [music] [1:43](https://www.youtube.com/watch?v=kTmUbRVsamo&t=103s) >> And that matters whether you're about to [1:45](https://www.youtube.com/watch?v=kTmUbRVsamo&t=105s) spend $900 or $40,000. [1:48](https://www.youtube.com/watch?v=kTmUbRVsamo&t=108s) >> [music] [1:48](https://www.youtube.com/watch?v=kTmUbRVsamo&t=108s) >> So, let's climb. Now, before we start, I [1:51](https://www.youtube.com/watch?v=kTmUbRVsamo&t=111s) want to explain why these tier videos [1:53](https://www.youtube.com/watch?v=kTmUbRVsamo&t=113s) feel so satisfying to watch, and why [1:56](https://www.youtube.com/watch?v=kTmUbRVsamo&t=116s) that feeling is a little bit of a trap. [1:58](https://www.youtube.com/watch?v=kTmUbRVsamo&t=118s) The case for buying more memory is [2:00](https://www.youtube.com/watch?v=kTmUbRVsamo&t=120s) really simple, and I understand why [2:01](https://www.youtube.com/watch?v=kTmUbRVsamo&t=121s) every video makes it. More memory holds [2:04](https://www.youtube.com/watch?v=kTmUbRVsamo&t=124s) a bigger model, and bigger models score [2:06](https://www.youtube.com/watch?v=kTmUbRVsamo&t=126s) higher on intelligence benchmarks. So, [2:09](https://www.youtube.com/watch?v=kTmUbRVsamo&t=129s) every step up the ladder gets us closer [2:11](https://www.youtube.com/watch?v=kTmUbRVsamo&t=131s) to the AI we're actually paying for when [2:13](https://www.youtube.com/watch?v=kTmUbRVsamo&t=133s) we use ChatGPT or Claude. And when we [2:15](https://www.youtube.com/watch?v=kTmUbRVsamo&t=135s) look at the benchmark scores, it seems [2:17](https://www.youtube.com/watch?v=kTmUbRVsamo&t=137s) to check out. On the Artificial Analysis [2:19](https://www.youtube.com/watch?v=kTmUbRVsamo&t=139s) Intelligence Index, which is the main [2:22](https://www.youtube.com/watch?v=kTmUbRVsamo&t=142s) benchmark people use for this, [2:24](https://www.youtube.com/watch?v=kTmUbRVsamo&t=144s) Claude Opus 5.5 at max effort scores [2:27](https://www.youtube.com/watch?v=kTmUbRVsamo&t=147s) 58.4 points, and that's the paid [2:29](https://www.youtube.com/watch?v=kTmUbRVsamo&t=149s) frontier. [2:31](https://www.youtube.com/watch?v=kTmUbRVsamo&t=151s) Quinn 3.8 27B, the model a 32 GB machine [2:35](https://www.youtube.com/watch?v=kTmUbRVsamo&t=155s) runs, scores 20.1, which is about a [2:38](https://www.youtube.com/watch?v=kTmUbRVsamo&t=158s) third of that. And DeepSeek V4.1 Flash, [2:42](https://www.youtube.com/watch?v=kTmUbRVsamo&t=162s) the model people buy 256 and 512 gig [2:45](https://www.youtube.com/watch?v=kTmUbRVsamo&t=165s) machines for, scores 32.5, which is [2:48](https://www.youtube.com/watch?v=kTmUbRVsamo&t=168s) about 56%. If we draw those as a line, [2:51](https://www.youtube.com/watch?v=kTmUbRVsamo&t=171s) it climbs from a third of the frontier [2:53](https://www.youtube.com/watch?v=kTmUbRVsamo&t=173s) at 32 gigs to more at 128, and more [2:56](https://www.youtube.com/watch?v=kTmUbRVsamo&t=176s) again at 256 and 512. [3:00](https://www.youtube.com/watch?v=kTmUbRVsamo&t=180s) And then it flattens. So, the advice [3:02](https://www.youtube.com/watch?v=kTmUbRVsamo&t=182s) seems to write itself, which is to stop [3:04](https://www.youtube.com/watch?v=kTmUbRVsamo&t=184s) at 512. [3:05](https://www.youtube.com/watch?v=kTmUbRVsamo&t=185s) But that line has three problems that [3:07](https://www.youtube.com/watch?v=kTmUbRVsamo&t=187s) the charts quietly smooth over. The [3:10](https://www.youtube.com/watch?v=kTmUbRVsamo&t=190s) first problem is that it averages across [3:12](https://www.youtube.com/watch?v=kTmUbRVsamo&t=192s) doublings where nothing changed. Because [3:14](https://www.youtube.com/watch?v=kTmUbRVsamo&t=194s) some of those steps on the ladder bought [3:16](https://www.youtube.com/watch?v=kTmUbRVsamo&t=196s) real intelligence, and others bought the [3:18](https://www.youtube.com/watch?v=kTmUbRVsamo&t=198s) same intelligence, just better stored. [3:20](https://www.youtube.com/watch?v=kTmUbRVsamo&t=200s) And when we flatten those together into [3:22](https://www.youtube.com/watch?v=kTmUbRVsamo&t=202s) one upward trend, it looks like every [3:24](https://www.youtube.com/watch?v=kTmUbRVsamo&t=204s) dollar we spend is getting us somewhere. [3:26](https://www.youtube.com/watch?v=kTmUbRVsamo&t=206s) The second problem is that several of [3:28](https://www.youtube.com/watch?v=kTmUbRVsamo&t=208s) those benchmark scores come from the old [3:31](https://www.youtube.com/watch?v=kTmUbRVsamo&t=211s) version of the index. The scores people [3:33](https://www.youtube.com/watch?v=kTmUbRVsamo&t=213s) are quoting this week for some models [3:35](https://www.youtube.com/watch?v=kTmUbRVsamo&t=215s) were measured before the index got its [3:37](https://www.youtube.com/watch?v=kTmUbRVsamo&t=217s) recent update, which changed things. And [3:40](https://www.youtube.com/watch?v=kTmUbRVsamo&t=220s) the third problem, the one that made me [3:41](https://www.youtube.com/watch?v=kTmUbRVsamo&t=221s) want to make this video, is that the [3:43](https://www.youtube.com/watch?v=kTmUbRVsamo&t=223s) entire chart shifted the week before [3:46](https://www.youtube.com/watch?v=kTmUbRVsamo&t=226s) those tier videos went up. A model [3:48](https://www.youtube.com/watch?v=kTmUbRVsamo&t=228s) landed on the terabyte rung that jumped [3:50](https://www.youtube.com/watch?v=kTmUbRVsamo&t=230s) 13.5 points above the previous ceiling. [3:53](https://www.youtube.com/watch?v=kTmUbRVsamo&t=233s) And in benchmark terms, that's a bigger [3:56](https://www.youtube.com/watch?v=kTmUbRVsamo&t=236s) leap than the last two rungs of the [3:57](https://www.youtube.com/watch?v=kTmUbRVsamo&t=237s) ladder combined. So, let me walk you [3:59](https://www.youtube.com/watch?v=kTmUbRVsamo&t=239s) through each rung, and I'll tell you [4:01](https://www.youtube.com/watch?v=kTmUbRVsamo&t=241s) exactly what each doubling actually [4:02](https://www.youtube.com/watch?v=kTmUbRVsamo&t=242s) buys. Now, I want to start at the [4:04](https://www.youtube.com/watch?v=kTmUbRVsamo&t=244s) bottom. This is actually where the most [4:06](https://www.youtube.com/watch?v=kTmUbRVsamo&t=246s) interesting value is hiding, and it's [4:08](https://www.youtube.com/watch?v=kTmUbRVsamo&t=248s) also the rung most people skip past in [4:10](https://www.youtube.com/watch?v=kTmUbRVsamo&t=250s) these videos because it feels unsexy. [4:13](https://www.youtube.com/watch?v=kTmUbRVsamo&t=253s) 16 GB is either an M-series Mac mini at [4:16](https://www.youtube.com/watch?v=kTmUbRVsamo&t=256s) around $899 [4:18](https://www.youtube.com/watch?v=kTmUbRVsamo&t=258s) or a 16 gig graphics card like the RTX [4:20](https://www.youtube.com/watch?v=kTmUbRVsamo&t=260s) 5060 Ti for about $780 plus whatever PC [4:25](https://www.youtube.com/watch?v=kTmUbRVsamo&t=265s) you're putting it in. And the best model [4:26](https://www.youtube.com/watch?v=kTmUbRVsamo&t=266s) a 16 gig setup can run is Qwen 3.8 27B, [4:31](https://www.youtube.com/watch?v=kTmUbRVsamo&t=271s) a 27 billion parameter model that as of [4:34](https://www.youtube.com/watch?v=kTmUbRVsamo&t=274s) right now scores 20.1 on the [4:36](https://www.youtube.com/watch?v=kTmUbRVsamo&t=276s) intelligence benchmark. At 4 bits of [4:39](https://www.youtube.com/watch?v=kTmUbRVsamo&t=279s) precision, that model weighs around 17 [4:41](https://www.youtube.com/watch?v=kTmUbRVsamo&t=281s) to 18 GB, which already doesn't quite [4:44](https://www.youtube.com/watch?v=kTmUbRVsamo&t=284s) fit in 16 once the machine starts [4:46](https://www.youtube.com/watch?v=kTmUbRVsamo&t=286s) holding a conversation. So, right away, [4:48](https://www.youtube.com/watch?v=kTmUbRVsamo&t=288s) the reality diverges from the clean [4:50](https://www.youtube.com/watch?v=kTmUbRVsamo&t=290s) story. On a 16 gig graphics card, owners [4:53](https://www.youtube.com/watch?v=kTmUbRVsamo&t=293s) run the 27B at 3 bits, which is a [4:56](https://www.youtube.com/watch?v=kTmUbRVsamo&t=296s) slightly compressed version that weighs [4:58](https://www.youtube.com/watch?v=kTmUbRVsamo&t=298s) 12 to 13 GB. And it works fine. [5:01](https://www.youtube.com/watch?v=kTmUbRVsamo&t=301s) One person on Local Lama measured their [5:03](https://www.youtube.com/watch?v=kTmUbRVsamo&t=303s) 5060 Ti at 45 tokens a second on short [5:06](https://www.youtube.com/watch?v=kTmUbRVsamo&t=306s) prompts and 31 tokens per second even [5:09](https://www.youtube.com/watch?v=kTmUbRVsamo&t=309s) with a 55,000 token context window, [5:12](https://www.youtube.com/watch?v=kTmUbRVsamo&t=312s) which is really fast for home hardware. [5:14](https://www.youtube.com/watch?v=kTmUbRVsamo&t=314s) But on a Mac, it's tighter because Mac [5:17](https://www.youtube.com/watch?v=kTmUbRVsamo&t=317s) OS gives the GPU about 3 quarters of [5:19](https://www.youtube.com/watch?v=kTmUbRVsamo&t=319s) available memory by default. So, a 16 [5:22](https://www.youtube.com/watch?v=kTmUbRVsamo&t=322s) gig Mac mini is actually working with [5:23](https://www.youtube.com/watch?v=kTmUbRVsamo&t=323s) around 12 gigs for the model. And at 12 [5:26](https://www.youtube.com/watch?v=kTmUbRVsamo&t=326s) gigs, we're running Google's Gemma 4 12B [5:29](https://www.youtube.com/watch?v=kTmUbRVsamo&t=329s) rather than the 27B, which is a smaller [5:31](https://www.youtube.com/watch?v=kTmUbRVsamo&t=331s) model with a lower score. And the few [5:33](https://www.youtube.com/watch?v=kTmUbRVsamo&t=333s) videos mention this kind of detail once [5:35](https://www.youtube.com/watch?v=kTmUbRVsamo&t=335s) and then move on. But it's actually [5:37](https://www.youtube.com/watch?v=kTmUbRVsamo&t=337s) really important because it changes the [5:39](https://www.youtube.com/watch?v=kTmUbRVsamo&t=339s) answer depending on what hardware we're [5:41](https://www.youtube.com/watch?v=kTmUbRVsamo&t=341s) looking at. Now, the step up to 32 GB, a [5:45](https://www.youtube.com/watch?v=kTmUbRVsamo&t=345s) $1,299 [5:47](https://www.youtube.com/watch?v=kTmUbRVsamo&t=347s) Mac mini or a 32 gig Radeon card at [5:49](https://www.youtube.com/watch?v=kTmUbRVsamo&t=349s) around $1,700, [5:51](https://www.youtube.com/watch?v=kTmUbRVsamo&t=351s) still runs the same 27B model with the [5:54](https://www.youtube.com/watch?v=kTmUbRVsamo&t=354s) same intelligence score of 20.1. But the [5:57](https://www.youtube.com/watch?v=kTmUbRVsamo&t=357s) model now has all four of its bits, [5:59](https://www.youtube.com/watch?v=kTmUbRVsamo&t=359s) [music] so it's running closer to its [6:01](https://www.youtube.com/watch?v=kTmUbRVsamo&t=361s) original quality. And more importantly, [6:03](https://www.youtube.com/watch?v=kTmUbRVsamo&t=363s) it has room for a 128,000 [6:06](https://www.youtube.com/watch?v=kTmUbRVsamo&t=366s) token context window. That might sound [6:09](https://www.youtube.com/watch?v=kTmUbRVsamo&t=369s) like a technical detail, but it's [6:11](https://www.youtube.com/watch?v=kTmUbRVsamo&t=371s) actually what separates AI that can [6:13](https://www.youtube.com/watch?v=kTmUbRVsamo&t=373s) answer questions from AI that can work [6:16](https://www.youtube.com/watch?v=kTmUbRVsamo&t=376s) through a complex coding project without [6:17](https://www.youtube.com/watch?v=kTmUbRVsamo&t=377s) forgetting what it was doing. So, the [6:19](https://www.youtube.com/watch?v=kTmUbRVsamo&t=379s) first doubling buys you bits and context [6:22](https://www.youtube.com/watch?v=kTmUbRVsamo&t=382s) rather than a smarter model, and [music] [6:23](https://www.youtube.com/watch?v=kTmUbRVsamo&t=383s) that's still worth knowing because [6:25](https://www.youtube.com/watch?v=kTmUbRVsamo&t=385s) context length matters a lot in [6:27](https://www.youtube.com/watch?v=kTmUbRVsamo&t=387s) practice. But, if you're already at 32 [6:29](https://www.youtube.com/watch?v=kTmUbRVsamo&t=389s) gigs and wondering whether 64 is worth [6:31](https://www.youtube.com/watch?v=kTmUbRVsamo&t=391s) it, I want to show you what actually [6:33](https://www.youtube.com/watch?v=kTmUbRVsamo&t=393s) happens. I'm going to call this the bits [6:35](https://www.youtube.com/watch?v=kTmUbRVsamo&t=395s) and vibes tier because that is really [6:38](https://www.youtube.com/watch?v=kTmUbRVsamo&t=398s) all it is. A 64 gig setup is the M5 Pro [6:41](https://www.youtube.com/watch?v=kTmUbRVsamo&t=401s) Mac Mini at $2,699 [6:45](https://www.youtube.com/watch?v=kTmUbRVsamo&t=405s) or two 32 [music] gig graphics cards. [6:47](https://www.youtube.com/watch?v=kTmUbRVsamo&t=407s) And after spending roughly double what [6:49](https://www.youtube.com/watch?v=kTmUbRVsamo&t=409s) the previous run cost, we are still [6:51](https://www.youtube.com/watch?v=kTmUbRVsamo&t=411s) running Qwen 3.827B [6:54](https://www.youtube.com/watch?v=kTmUbRVsamo&t=414s) at 20.1 on the benchmark, which is still [6:57](https://www.youtube.com/watch?v=kTmUbRVsamo&t=417s) the same AI. So, what actually changed? [7:00](https://www.youtube.com/watch?v=kTmUbRVsamo&t=420s) The model now runs at 8 bits, which is [7:02](https://www.youtube.com/watch?v=kTmUbRVsamo&t=422s) close enough to the original that you [7:04](https://www.youtube.com/watch?v=kTmUbRVsamo&t=424s) would struggle to tell the difference in [7:05](https://www.youtube.com/watch?v=kTmUbRVsamo&t=425s) a side-by-side test, [music] [7:07](https://www.youtube.com/watch?v=kTmUbRVsamo&t=427s) and we now have room for the full [7:08](https://www.youtube.com/watch?v=kTmUbRVsamo&t=428s) 256,000 [7:10](https://www.youtube.com/watch?v=kTmUbRVsamo&t=430s) token context window. The whole setup, [7:13](https://www.youtube.com/watch?v=kTmUbRVsamo&t=433s) 8-bit weights plus the full context, [7:15](https://www.youtube.com/watch?v=kTmUbRVsamo&t=435s) comes to around 46 GB, [7:17](https://www.youtube.com/watch?v=kTmUbRVsamo&t=437s) so it fits cleanly. There is one [7:19](https://www.youtube.com/watch?v=kTmUbRVsamo&t=439s) underrated thing about this run though, [7:21](https://www.youtube.com/watch?v=kTmUbRVsamo&t=441s) that the tier charts almost completely [7:24](https://www.youtube.com/watch?v=kTmUbRVsamo&t=444s) [music] ignore, and that's concurrency. [7:26](https://www.youtube.com/watch?v=kTmUbRVsamo&t=446s) The weights for a model load into memory [7:28](https://www.youtube.com/watch?v=kTmUbRVsamo&t=448s) [music] once, [7:29](https://www.youtube.com/watch?v=kTmUbRVsamo&t=449s) but if we're running multiple AI agents [7:31](https://www.youtube.com/watch?v=kTmUbRVsamo&t=451s) at the same time, say one writing code [7:34](https://www.youtube.com/watch?v=kTmUbRVsamo&t=454s) while another summarizes documents, each [7:36](https://www.youtube.com/watch?v=kTmUbRVsamo&t=456s) agent keeps its own separate memory of [7:39](https://www.youtube.com/watch?v=kTmUbRVsamo&t=459s) the conversation, called the KV [music] [7:41](https://www.youtube.com/watch?v=kTmUbRVsamo&t=461s) cache. And one owner reported running up [7:43](https://www.youtube.com/watch?v=kTmUbRVsamo&t=463s) to 10 simultaneous sessions on the 27B [7:47](https://www.youtube.com/watch?v=kTmUbRVsamo&t=467s) at this tier, clocking around 150 tokens [7:50](https://www.youtube.com/watch?v=kTmUbRVsamo&t=470s) per second combined across all of them. [7:52](https://www.youtube.com/watch?v=kTmUbRVsamo&t=472s) >> [music] [7:52](https://www.youtube.com/watch?v=kTmUbRVsamo&t=472s) >> So, if you're building something that [7:53](https://www.youtube.com/watch?v=kTmUbRVsamo&t=473s) runs multiple agents, this rung matters [7:56](https://www.youtube.com/watch?v=kTmUbRVsamo&t=476s) more than the benchmark score suggests, [7:58](https://www.youtube.com/watch?v=kTmUbRVsamo&t=478s) and if you're one person running one [8:00](https://www.youtube.com/watch?v=kTmUbRVsamo&t=480s) agent, it's a tier you can probably [8:02](https://www.youtube.com/watch?v=kTmUbRVsamo&t=482s) skip. Speed-wise, it's measured at about [8:04](https://www.youtube.com/watch?v=kTmUbRVsamo&t=484s) 10 tokens a second on the 64 gig M5 Pro [8:08](https://www.youtube.com/watch?v=kTmUbRVsamo&t=488s) Mini. [8:08](https://www.youtube.com/watch?v=kTmUbRVsamo&t=488s) That's fast enough for normal use, but [8:11](https://www.youtube.com/watch?v=kTmUbRVsamo&t=491s) not the jump you were hoping for. And my [8:13](https://www.youtube.com/watch?v=kTmUbRVsamo&t=493s) honest take is that if you're sitting at [8:14](https://www.youtube.com/watch?v=kTmUbRVsamo&t=494s) 32 gigs and thinking about upgrading, [8:16](https://www.youtube.com/watch?v=kTmUbRVsamo&t=496s) you should skip 64 entirely unless you [8:19](https://www.youtube.com/watch?v=kTmUbRVsamo&t=499s) have a specific multi-agent use case. [8:21](https://www.youtube.com/watch?v=kTmUbRVsamo&t=501s) The next rung is where things get [8:23](https://www.youtube.com/watch?v=kTmUbRVsamo&t=503s) interesting. Okay, now the ladder [8:25](https://www.youtube.com/watch?v=kTmUbRVsamo&t=505s) actually moves. The 128-gigabyte tier is [8:28](https://www.youtube.com/watch?v=kTmUbRVsamo&t=508s) the first doubling that buys us a [8:29](https://www.youtube.com/watch?v=kTmUbRVsamo&t=509s) smaller model, and it's also the tier [8:31](https://www.youtube.com/watch?v=kTmUbRVsamo&t=511s) with the most booby traps. So, let's [8:33](https://www.youtube.com/watch?v=kTmUbRVsamo&t=513s) look at it carefully. [8:35](https://www.youtube.com/watch?v=kTmUbRVsamo&t=515s) The hardware options here are a [8:36](https://www.youtube.com/watch?v=kTmUbRVsamo&t=516s) Framework AMD Strix Halo desktop [8:38](https://www.youtube.com/watch?v=kTmUbRVsamo&t=518s) starting at around $3,450, [8:41](https://www.youtube.com/watch?v=kTmUbRVsamo&t=521s) the Mac Studio with M5 Max at $5,099, [8:45](https://www.youtube.com/watch?v=kTmUbRVsamo&t=525s) and Nvidia's [music] DGX Spark at [8:47](https://www.youtube.com/watch?v=kTmUbRVsamo&t=527s) $6,950. [8:49](https://www.youtube.com/watch?v=kTmUbRVsamo&t=529s) And that last one doubled in price from [8:51](https://www.youtube.com/watch?v=kTmUbRVsamo&t=531s) a year ago when it was $3,999. [8:55](https://www.youtube.com/watch?v=kTmUbRVsamo&t=535s) And the model people buy 128 gigs for is [8:58](https://www.youtube.com/watch?v=kTmUbRVsamo&t=538s) Qwen 3.8 FlashNext, which has 125 [9:02](https://www.youtube.com/watch?v=kTmUbRVsamo&t=542s) billion parameters, and Unsloth's 4-bit [9:05](https://www.youtube.com/watch?v=kTmUbRVsamo&t=545s) build of it weighs about 111 gigabytes. [9:08](https://www.youtube.com/watch?v=kTmUbRVsamo&t=548s) The first trap is that the current [9:09](https://www.youtube.com/watch?v=kTmUbRVsamo&t=549s) version of the benchmark index everyone [9:11](https://www.youtube.com/watch?v=kTmUbRVsamo&t=551s) is citing this week [music] has not [9:13](https://www.youtube.com/watch?v=kTmUbRVsamo&t=553s) scored FlashNext yet. So, any percentage [9:16](https://www.youtube.com/watch?v=kTmUbRVsamo&t=556s) improvement you've seen quoted for this [9:18](https://www.youtube.com/watch?v=kTmUbRVsamo&t=558s) model in a recent video did not come [9:20](https://www.youtube.com/watch?v=kTmUbRVsamo&t=560s) from the current index. It came from an [9:22](https://www.youtube.com/watch?v=kTmUbRVsamo&t=562s) older score on an older test. That [9:24](https://www.youtube.com/watch?v=kTmUbRVsamo&t=564s) doesn't mean the model is bad, [music] [9:26](https://www.youtube.com/watch?v=kTmUbRVsamo&t=566s) it just means the comparison is harder [9:28](https://www.youtube.com/watch?v=kTmUbRVsamo&t=568s) to make cleanly. Now, the model at this [9:30](https://www.youtube.com/watch?v=kTmUbRVsamo&t=570s) tier that does have a fresh score is GLM [9:34](https://www.youtube.com/watch?v=kTmUbRVsamo&t=574s) 5.3 Flash [music] [9:35](https://www.youtube.com/watch?v=kTmUbRVsamo&t=575s) at 320 billion total parameters with [9:38](https://www.youtube.com/watch?v=kTmUbRVsamo&t=578s) only 18 billion active, and it scores [9:41](https://www.youtube.com/watch?v=kTmUbRVsamo&t=581s) 27.8. [9:42](https://www.youtube.com/watch?v=kTmUbRVsamo&t=582s) That's a real step up from the 20.1 at [9:45](https://www.youtube.com/watch?v=kTmUbRVsamo&t=585s) the bottom of the ladder, and it's the [9:46](https://www.youtube.com/watch?v=kTmUbRVsamo&t=586s) first meaningful intelligence gain on [9:48](https://www.youtube.com/watch?v=kTmUbRVsamo&t=588s) the whole chart. The second trap [9:50](https://www.youtube.com/watch?v=kTmUbRVsamo&t=590s) specifically bit Mac owners [9:52](https://www.youtube.com/watch?v=kTmUbRVsamo&t=592s) >> [music] [9:53](https://www.youtube.com/watch?v=kTmUbRVsamo&t=593s) >> because 3/4 of 128 gigabytes is about [9:55](https://www.youtube.com/watch?v=kTmUbRVsamo&t=595s) 103 gigs of usable GPU memory with macOS [9:58](https://www.youtube.com/watch?v=kTmUbRVsamo&t=598s) defaults, and [music] the 4-bit [10:00](https://www.youtube.com/watch?v=kTmUbRVsamo&t=600s) FlashNext build is 111 gigs. So, out of [10:04](https://www.youtube.com/watch?v=kTmUbRVsamo&t=604s) the box on a Mac, you are not running [10:07](https://www.youtube.com/watch?v=kTmUbRVsamo&t=607s) 111-gig model, you're running the [10:09](https://www.youtube.com/watch?v=kTmUbRVsamo&t=609s) smaller 93-gig version, and you won't [10:11](https://www.youtube.com/watch?v=kTmUbRVsamo&t=611s) even know it unless you go looking. [10:13](https://www.youtube.com/watch?v=kTmUbRVsamo&t=613s) There is a single terminal command that [10:16](https://www.youtube.com/watch?v=kTmUbRVsamo&t=616s) raises the GPU memory limit, and that [10:18](https://www.youtube.com/watch?v=kTmUbRVsamo&t=618s) one command is the difference between [10:20](https://www.youtube.com/watch?v=kTmUbRVsamo&t=620s) running the full model and running the [10:22](https://www.youtube.com/watch?v=kTmUbRVsamo&t=622s) cutdown version on the same $5,000 [10:24](https://www.youtube.com/watch?v=kTmUbRVsamo&t=624s) machine. I'll be honest, I only found [10:26](https://www.youtube.com/watch?v=kTmUbRVsamo&t=626s) out about it because someone complained [10:28](https://www.youtube.com/watch?v=kTmUbRVsamo&t=628s) in the comments of a YouTube video at [10:30](https://www.youtube.com/watch?v=kTmUbRVsamo&t=630s) 2:00 a.m. and I happened to be [10:31](https://www.youtube.com/watch?v=kTmUbRVsamo&t=631s) scrolling. This is not information that [10:33](https://www.youtube.com/watch?v=kTmUbRVsamo&t=633s) tier videos lead with. In speed at this [10:35](https://www.youtube.com/watch?v=kTmUbRVsamo&t=635s) run is solid. Flash next runs around 69 [10:39](https://www.youtube.com/watch?v=kTmUbRVsamo&t=639s) a second on the M5 Max and [music] 47 to [10:41](https://www.youtube.com/watch?v=kTmUbRVsamo&t=641s) 54 on the Strix Halo machine with the [10:43](https://www.youtube.com/watch?v=kTmUbRVsamo&t=643s) newer Strada engine. So, the third [10:45](https://www.youtube.com/watch?v=kTmUbRVsamo&t=645s) doubling is the first one that actually [10:47](https://www.youtube.com/watch?v=kTmUbRVsamo&t=647s) buys you smarter AI with a jump from [10:50](https://www.youtube.com/watch?v=kTmUbRVsamo&t=650s) 20.1 to 27.8. You take it at three-bit [10:54](https://www.youtube.com/watch?v=kTmUbRVsamo&t=654s) precision, where the model agrees with [10:56](https://www.youtube.com/watch?v=kTmUbRVsamo&t=656s) its own full precision version about [10:58](https://www.youtube.com/watch?v=kTmUbRVsamo&t=658s) 81.6% of the time, which is one [11:00](https://www.youtube.com/watch?v=kTmUbRVsamo&t=660s) different word choice every five tokens. [11:03](https://www.youtube.com/watch?v=kTmUbRVsamo&t=663s) And in practice, for most tasks, that's [11:06](https://www.youtube.com/watch?v=kTmUbRVsamo&t=666s) invisible. Now, a lot of serious home AI [11:08](https://www.youtube.com/watch?v=kTmUbRVsamo&t=668s) people have been anchoring around the [11:10](https://www.youtube.com/watch?v=kTmUbRVsamo&t=670s) next run because 256 is where DeepSeek [11:13](https://www.youtube.com/watch?v=kTmUbRVsamo&t=673s) V4.1 Flash first shows up. And DeepSeek [11:17](https://www.youtube.com/watch?v=kTmUbRVsamo&t=677s) is a really different kind of model. [11:18](https://www.youtube.com/watch?v=kTmUbRVsamo&t=678s) [music] A 256 gig setup is a Mac Studio [11:21](https://www.youtube.com/watch?v=kTmUbRVsamo&t=681s) with M5 Ultra at $9,499 [11:25](https://www.youtube.com/watch?v=kTmUbRVsamo&t=685s) or two DGX Sparks for around $13,900. [11:29](https://www.youtube.com/watch?v=kTmUbRVsamo&t=689s) DeepSeek V4.1 Flash is 552 [music] [11:33](https://www.youtube.com/watch?v=kTmUbRVsamo&t=693s) billion total parameters, and it scores [11:35](https://www.youtube.com/watch?v=kTmUbRVsamo&t=695s) 32.5 on the benchmark, which is another [11:37](https://www.youtube.com/watch?v=kTmUbRVsamo&t=697s) step up from the 27.8 at 128 gigs. What [11:41](https://www.youtube.com/watch?v=kTmUbRVsamo&t=701s) makes DeepSeek unusual is the way it [11:43](https://www.youtube.com/watch?v=kTmUbRVsamo&t=703s) handles memory. It comes with a lookup [11:45](https://www.youtube.com/watch?v=kTmUbRVsamo&t=705s) table that DeepSeek calls Ingram, and [11:48](https://www.youtube.com/watch?v=kTmUbRVsamo&t=708s) that table is about 196 billion [11:50](https://www.youtube.com/watch?v=kTmUbRVsamo&t=710s) parameters on its own. And the smart [11:52](https://www.youtube.com/watch?v=kTmUbRVsamo&t=712s) part is that the inference engines leave [11:54](https://www.youtube.com/watch?v=kTmUbRVsamo&t=714s) Ingram on your SSD and stream it in as [11:56](https://www.youtube.com/watch?v=kTmUbRVsamo&t=716s) needed. So, we don't have to fit it all [11:58](https://www.youtube.com/watch?v=kTmUbRVsamo&t=718s) in fast memory, and only the rest needs [12:00](https://www.youtube.com/watch?v=kTmUbRVsamo&t=720s) to sit in RAM. Unsloth's three-bit build [12:03](https://www.youtube.com/watch?v=kTmUbRVsamo&t=723s) of the non-Ingram parts is 233.5 gigs, [12:06](https://www.youtube.com/watch?v=kTmUbRVsamo&t=726s) and at three bits, it agrees with the [12:08](https://www.youtube.com/watch?v=kTmUbRVsamo&t=728s) original 95.1% [music] of the time, [12:11](https://www.youtube.com/watch?v=kTmUbRVsamo&t=731s) which is one different bit about every [12:12](https://www.youtube.com/watch?v=kTmUbRVsamo&t=732s) 20 tokens. And that's pretty clean, [12:15](https://www.youtube.com/watch?v=kTmUbRVsamo&t=735s) because at 256 gigs, we're getting a [12:17](https://www.youtube.com/watch?v=kTmUbRVsamo&t=737s) more capable model running in a way that [12:19](https://www.youtube.com/watch?v=kTmUbRVsamo&t=739s) was specifically engineered to fit this [12:21](https://www.youtube.com/watch?v=kTmUbRVsamo&t=741s) kind of hardware. Except when we look at [12:23](https://www.youtube.com/watch?v=kTmUbRVsamo&t=743s) a Mac, the default memory split comes [12:25](https://www.youtube.com/watch?v=kTmUbRVsamo&t=745s) into play again. 3/4 of 256 is about 206 [12:29](https://www.youtube.com/watch?v=kTmUbRVsamo&t=749s) gigs for the GPU. So, JLM 5.3 flash at [12:33](https://www.youtube.com/watch?v=kTmUbRVsamo&t=753s) 4-bits at 199.7 [12:35](https://www.youtube.com/watch?v=kTmUbRVsamo&t=755s) gigs fits with room to spare. And deep [12:38](https://www.youtube.com/watch?v=kTmUbRVsamo&t=758s) seek at 3-bits at 233.5 [12:41](https://www.youtube.com/watch?v=kTmUbRVsamo&t=761s) doesn't fit. And that's the same box at [12:43](https://www.youtube.com/watch?v=kTmUbRVsamo&t=763s) the same price where one command in [12:45](https://www.youtube.com/watch?v=kTmUbRVsamo&t=765s) terminal is the difference between a [12:47](https://www.youtube.com/watch?v=kTmUbRVsamo&t=767s) 27.8 and a 32.5. I saw someone in the [12:50](https://www.youtube.com/watch?v=kTmUbRVsamo&t=770s) comments of a tier video ask whether [12:52](https://www.youtube.com/watch?v=kTmUbRVsamo&t=772s) [music] 256 was worth upgrading to from [12:55](https://www.youtube.com/watch?v=kTmUbRVsamo&t=775s) 128 for what looked like a 2% gain. The [12:59](https://www.youtube.com/watch?v=kTmUbRVsamo&t=779s) answer they got was probably not, but [13:01](https://www.youtube.com/watch?v=kTmUbRVsamo&t=781s) what the answer should have been is that [13:03](https://www.youtube.com/watch?v=kTmUbRVsamo&t=783s) depends entirely on whether you know [13:04](https://www.youtube.com/watch?v=kTmUbRVsamo&t=784s) about this terminal setting. Writing [13:06](https://www.youtube.com/watch?v=kTmUbRVsamo&t=786s) speed at this rung runs from about 26 to [13:09](https://www.youtube.com/watch?v=kTmUbRVsamo&t=789s) 35 tokens per second [music] on the Mac. [13:11](https://www.youtube.com/watch?v=kTmUbRVsamo&t=791s) And reading speed really matters if [13:13](https://www.youtube.com/watch?v=kTmUbRVsamo&t=793s) you're running AI agents, because they [13:15](https://www.youtube.com/watch?v=kTmUbRVsamo&t=795s) send long prompts on every step, and it [13:18](https://www.youtube.com/watch?v=kTmUbRVsamo&t=798s) varies wildly by which engine [music] [13:19](https://www.youtube.com/watch?v=kTmUbRVsamo&t=799s) you use. The same Mac measures somewhere [13:22](https://www.youtube.com/watch?v=kTmUbRVsamo&t=802s) between 18 and 80 seconds to read a [13:24](https://www.youtube.com/watch?v=kTmUbRVsamo&t=804s) 32,000 [music] token prompt, depending [13:26](https://www.youtube.com/watch?v=kTmUbRVsamo&t=806s) on software, while two DGX sparks handle [13:29](https://www.youtube.com/watch?v=kTmUbRVsamo&t=809s) that in about 15 seconds. So, that [13:31](https://www.youtube.com/watch?v=kTmUbRVsamo&t=811s) prompt reading gap is real, and if [13:33](https://www.youtube.com/watch?v=kTmUbRVsamo&t=813s) you're doing agent work, the engine you [13:35](https://www.youtube.com/watch?v=kTmUbRVsamo&t=815s) pick at this tier matters as much as the [13:37](https://www.youtube.com/watch?v=kTmUbRVsamo&t=817s) hardware. Now, the 512 gig Mac Studio [13:41](https://www.youtube.com/watch?v=kTmUbRVsamo&t=821s) went on sale on October 28th at $22,299, [13:46](https://www.youtube.com/watch?v=kTmUbRVsamo&t=826s) and the 256 gig Mac Studio costs $9,499. [13:51](https://www.youtube.com/watch?v=kTmUbRVsamo&t=831s) So, that's $11,500 [13:53](https://www.youtube.com/watch?v=kTmUbRVsamo&t=833s) more, and what does that money actually [13:55](https://www.youtube.com/watch?v=kTmUbRVsamo&t=835s) buy? [13:56](https://www.youtube.com/watch?v=kTmUbRVsamo&t=836s) At 512 gigs, deep seek runs at 4-bits [14:00](https://www.youtube.com/watch?v=kTmUbRVsamo&t=840s) instead of 3. The 4-bit version weighs [14:02](https://www.youtube.com/watch?v=kTmUbRVsamo&t=842s) about 298 gigs and agrees with the [14:05](https://www.youtube.com/watch?v=kTmUbRVsamo&t=845s) original 98.5% of the time, which is one [14:09](https://www.youtube.com/watch?v=kTmUbRVsamo&t=849s) different pick every 67 tokens instead [14:11](https://www.youtube.com/watch?v=kTmUbRVsamo&t=851s) of every 20. That's cleaner output, and [14:14](https://www.youtube.com/watch?v=kTmUbRVsamo&t=854s) for long complex reasoning tasks, those [14:17](https://www.youtube.com/watch?v=kTmUbRVsamo&t=857s) accumulated differences can add up. But, [14:19](https://www.youtube.com/watch?v=kTmUbRVsamo&t=859s) the intelligence score is the same [14:21](https://www.youtube.com/watch?v=kTmUbRVsamo&t=861s) model's 32.5. And, the other big open [14:24](https://www.youtube.com/watch?v=kTmUbRVsamo&t=864s) model that fits here, GLM 5.3 at 753 [14:28](https://www.youtube.com/watch?v=kTmUbRVsamo&t=868s) billion parameters, actually scores [14:30](https://www.youtube.com/watch?v=kTmUbRVsamo&t=870s) 31.7, which is slightly lower despite it [14:33](https://www.youtube.com/watch?v=kTmUbRVsamo&t=873s) being bigger. [14:34](https://www.youtube.com/watch?v=kTmUbRVsamo&t=874s) So, this rung is a really clean [14:36](https://www.youtube.com/watch?v=kTmUbRVsamo&t=876s) illustration that more parameters don't [14:39](https://www.youtube.com/watch?v=kTmUbRVsamo&t=879s) always mean more intelligence. If you [14:41](https://www.youtube.com/watch?v=kTmUbRVsamo&t=881s) want 512 gigs of useful RAM, the local [14:44](https://www.youtube.com/watch?v=kTmUbRVsamo&t=884s) Llama community figured out a piece of [14:46](https://www.youtube.com/watch?v=kTmUbRVsamo&t=886s) math pretty fast. Two 256 gig Mac [14:50](https://www.youtube.com/watch?v=kTmUbRVsamo&t=890s) Studios on the same chip are $21,598, [14:55](https://www.youtube.com/watch?v=kTmUbRVsamo&t=895s) which is $701 less than one 512 gig box, [14:59](https://www.youtube.com/watch?v=kTmUbRVsamo&t=899s) and they link together over a single [15:01](https://www.youtube.com/watch?v=kTmUbRVsamo&t=901s) Thunderbolt 5 cable. The top thread [15:04](https://www.youtube.com/watch?v=kTmUbRVsamo&t=904s) about the 512's price on that subreddit [15:06](https://www.youtube.com/watch?v=kTmUbRVsamo&t=906s) was basically just people doing that [15:08](https://www.youtube.com/watch?v=kTmUbRVsamo&t=908s) subtraction in the comments. And, [15:09](https://www.youtube.com/watch?v=kTmUbRVsamo&t=909s) there's also a wilder option where [15:11](https://www.youtube.com/watch?v=kTmUbRVsamo&t=911s) someone on X is running Deep Seek's [15:13](https://www.youtube.com/watch?v=kTmUbRVsamo&t=913s) original weights at 65 tokens per second [15:15](https://www.youtube.com/watch?v=kTmUbRVsamo&t=915s) using a single RTX PRO 6000 graphics [15:18](https://www.youtube.com/watch?v=kTmUbRVsamo&t=918s) card with 512 gigs of regular DDR4 [15:22](https://www.youtube.com/watch?v=kTmUbRVsamo&t=922s) server RAM attached to a pair of 2019 [15:24](https://www.youtube.com/watch?v=kTmUbRVsamo&t=924s) server chips. Old server memory is [15:26](https://www.youtube.com/watch?v=kTmUbRVsamo&t=926s) extremely cheap right now at about $399 [15:30](https://www.youtube.com/watch?v=kTmUbRVsamo&t=930s) for 64 gigs compared to over $1,000 for [15:33](https://www.youtube.com/watch?v=kTmUbRVsamo&t=933s) the same amount of current gen DDR5. [15:35](https://www.youtube.com/watch?v=kTmUbRVsamo&t=935s) That kind of setup is not a polished [15:37](https://www.youtube.com/watch?v=kTmUbRVsamo&t=937s) consumer experience, but it gets the [15:39](https://www.youtube.com/watch?v=kTmUbRVsamo&t=939s) model running. So, the fifth doubling [15:41](https://www.youtube.com/watch?v=kTmUbRVsamo&t=941s) from 256 to 512 buys Deep Seek's fourth [15:45](https://www.youtube.com/watch?v=kTmUbRVsamo&t=945s) bit. And, if you need that precision, [15:47](https://www.youtube.com/watch?v=kTmUbRVsamo&t=947s) buy it as two 256s because that's the [15:50](https://www.youtube.com/watch?v=kTmUbRVsamo&t=950s) same model and the same experience for [15:52](https://www.youtube.com/watch?v=kTmUbRVsamo&t=952s) less money. Now, we get to the rung that [15:54](https://www.youtube.com/watch?v=kTmUbRVsamo&t=954s) every tier video this week called a dead [15:57](https://www.youtube.com/watch?v=kTmUbRVsamo&t=957s) end. And, I want to be clear that as of [15:59](https://www.youtube.com/watch?v=kTmUbRVsamo&t=959s) 6 days ago, they were right. A terabyte [16:02](https://www.youtube.com/watch?v=kTmUbRVsamo&t=962s) of RAM runs the same models that 512 [16:04](https://www.youtube.com/watch?v=kTmUbRVsamo&t=964s) gigs runs, just with more bits. So, [16:06](https://www.youtube.com/watch?v=kTmUbRVsamo&t=966s) that's Deep Seek at 8 bits and GLM 5.3 [16:09](https://www.youtube.com/watch?v=kTmUbRVsamo&t=969s) the way it shipped. Mistral Large 4, [16:12](https://www.youtube.com/watch?v=kTmUbRVsamo&t=972s) which actually went open this week, [16:13](https://www.youtube.com/watch?v=kTmUbRVsamo&t=973s) [music] weighs over a terabyte on its [16:15](https://www.youtube.com/watch?v=kTmUbRVsamo&t=975s) own and scores 26, which is below [16:18](https://www.youtube.com/watch?v=kTmUbRVsamo&t=978s) DeepSeek. [music] And it gets weirder [16:20](https://www.youtube.com/watch?v=kTmUbRVsamo&t=980s) because Inkling, a model at nearly a [16:22](https://www.youtube.com/watch?v=kTmUbRVsamo&t=982s) trillion parameters, scores 12.5. [16:24](https://www.youtube.com/watch?v=kTmUbRVsamo&t=984s) [music] [16:25](https://www.youtube.com/watch?v=kTmUbRVsamo&t=985s) And Nvidia's NeMo tron 3 ultra at 550 [16:28](https://www.youtube.com/watch?v=kTmUbRVsamo&t=988s) billion parameters scores 11.8. Both of [16:32](https://www.youtube.com/watch?v=kTmUbRVsamo&t=992s) them score lower than Qwen 3.8 27B, the [16:35](https://www.youtube.com/watch?v=kTmUbRVsamo&t=995s) model that runs on a $1,300 machine. So, [16:38](https://www.youtube.com/watch?v=kTmUbRVsamo&t=998s) bigger is not smarter, and that pattern [16:40](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1000s) [music] repeats all the way up the [16:41](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1001s) ladder. But, it's never more obvious [16:43](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1003s) than at the terabyte tier. But, on [16:45](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1005s) October 23rd, [music] a Chinese AI lab [16:48](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1008s) called Moonshot launched a model called [16:50](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1010s) Kimi K3.1 Pro on their API. [16:54](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1014s) On the new benchmarking index, it scores [16:56](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1016s) 46.0, [16:57](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1017s) which is the highest score of any model [16:59](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1019s) with weights coming. That's 13.5 points [17:02](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1022s) above DeepSeek. To put that in context, [17:05](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1025s) the entire gap between the 32 gig run [17:08](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1028s) [music] and the 256 gig run is about 12 [17:11](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1031s) points. So, this single model clears all [17:14](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1034s) of that and then some. And the weights [17:16](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1036s) are promised for November 4th, which is [17:18](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1038s) 2 days from when I'm making this video. [17:20](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1040s) >> [music] [17:20](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1040s) >> Kimi K3.1 Pro is 2.8 trillion total [17:24](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1044s) parameters with 104 billion active. And [17:27](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1047s) because its architecture is the same as [17:29](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1049s) the July Kimi K3, the full file is [17:32](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1052s) expected to weigh about 1.56 [music] [17:34](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1054s) terabytes, which doesn't fit in 1 [17:36](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1056s) terabyte. And this is where the [17:38](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1058s) compression [music] numbers come in. [17:40](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1060s) Unsloth, the team that builds quantized [17:42](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1062s) versions of big models, has already done [17:45](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1065s) this with the earlier Kimi K3, [music] [17:47](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1067s) where their 2-bit build comes in at 861 [17:50](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1070s) GB, and their smaller [music] 2-bit [17:52](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1072s) build is 711 GB. [17:54](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1074s) The 861 gig version agrees with the [17:57](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1077s) original 90.4% of the time, and the 711 [18:01](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1081s) gig version agrees 84.1%, [18:04](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1084s) which works out to one different word [18:06](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1086s) choice roughly every 6 to 10 tokens. [18:08](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1088s) Four 256 gig Mac Studios add up to about [18:12](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1092s) 1,100 gigs of memory, and at the default [18:15](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1095s) Mac memory split, the GPUs get around [18:17](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1097s) 825 gigs between them, which is enough [18:20](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1100s) for the 711 gig build. And if we raise [18:24](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1104s) the memory limit on all four machines, [18:26](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1106s) the 861 fits, too. So, the sixth [18:29](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1109s) doubling, from 512 to a terabyte, just [18:32](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1112s) became the biggest single leap on the [18:34](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1114s) entire ladder. And it's not a small [18:37](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1117s) precision improvement, it's a 13.5 point [18:40](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1120s) intelligence jump to the best open model [18:43](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1123s) [music] in the world. Nobody has scored [18:45](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1125s) the two-bit version yet, because nobody [18:47](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1127s) outside Moonshot has the weights, and [18:49](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1129s) that's the honest caveat, but the [18:51](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1131s) direction is clear. Now, I want to pause [18:54](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1134s) on the ladder for a second and talk [18:56](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1136s) about a path that doesn't fit neatly on [18:58](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1138s) any [music] of these tier charts. It has [19:00](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1140s) really changed what's possible on a PC [19:02](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1142s) this fall. Every rung we've climbed so [19:04](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1144s) far treats memory as one pool, where we [19:06](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1146s) either have enough RAM for the model or [19:08](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1148s) we don't. But on a desktop PC, we [19:11](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1151s) actually have two pools, fast memory on [19:13](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1153s) the graphics card and slower system RAM. [19:16](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1156s) And this fall, [music] an open-source [19:17](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1157s) engine called Strata figured out how to [19:19](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1159s) use both at once. The way it works is [19:22](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1162s) that Strata keeps the parts of the model [19:24](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1164s) that every [music] token needs on the [19:25](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1165s) graphics card and puts the rest in [19:27](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1167s) system RAM, while the lookup tables stay [19:29](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1169s) on the SSD. And on an RTX 5070 with 12 [19:33](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1173s) gigs of VRAM and 64 gigs of system RAM, [19:36](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1176s) its maintainers measured a two-bit [19:37](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1177s) version of Flash Next writing 94 tokens [19:40](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1180s) per second. That's a 111 gig model [19:43](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1183s) running on a machine with 76 gigs of [19:45](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1185s) [music] memory. It works for DeepSeek, [19:47](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1187s) too, though more slowly, with the same [19:49](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1189s) 5070 box writing four to six tokens per [19:52](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1192s) second on DeepSeek, while an older RTX [19:55](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1195s) 3060 box manages nine to 11 tokens per [19:58](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1198s) second. Which is interesting, because it [19:59](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1199s) means a cheaper GPU with slower memory [20:02](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1202s) actually performs better in this [20:03](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1203s) configuration. The catch to know about [20:06](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1206s) is context. Strata's default context [20:08](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1208s) window is 8,192 [20:11](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1211s) tokens, which is actually smaller than [20:13](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1213s) some AI agent system prompts, so that's [20:15](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1215s) a setting you need to adjust before [20:17](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1217s) using it for anything serious. But for [20:19](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1219s) PC builders, the main implication is [20:21](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1221s) that when we're sizing a system, the [20:24](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1224s) number that matters isn't just RAM or [20:26](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1226s) just VRAM, [music] [20:27](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1227s) it's the two combined. And right now, [20:30](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1230s) because old server memory has gotten [20:31](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1231s) cheap at about $399 per 64 gigs versus [20:35](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1235s) $1,200 for the equivalent in new DDR5, [20:39](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1239s) we can build a surprisingly capable [20:41](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1241s) machine for much less than the Mac route [20:44](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1244s) would suggest. [20:45](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1245s) Now, the thing about all of this that [20:47](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1247s) blew my mind the first time I understood [20:49](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1249s) [music] it is that every step up the [20:51](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1251s) memory ladder costs more to own. But [20:54](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1254s) renting those same models runs in the [20:56](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1256s) exact opposite direction. On OpenRouter, [20:59](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1259s) as of November 1st, Qwen 3.8 27B, the [21:02](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1262s) model you run on a $1,300 machine costs [21:06](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1266s) $1.875 [21:08](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1268s) per million output tokens. And GLM 5.3 [21:12](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1272s) Flash, the model you buy a 128 gig [21:15](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1275s) machine to run locally, costs about 9.2 [21:18](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1278s) cents per million tokens. So, the model [21:20](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1280s) that needs eight times more memory to [21:22](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1282s) run costs about 20 times less to rent. [21:26](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1286s) The models that are expensive to own [21:27](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1287s) turn out to be the cheap ones to rent, [21:29](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1289s) and the other way around. [21:31](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1291s) So, why does renting run backwards? [21:33](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1293s) It comes down to a number we talked [21:35](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1295s) about earlier, which is active [21:36](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1296s) parameters. When we buy hardware, the [21:39](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1299s) total size of the model is what [21:41](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1301s) determines which machine we need. But [21:43](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1303s) when a hosting company charges us, they [21:45](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1305s) charge for the work the model actually [21:47](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1307s) does, and that's the active parameters. [21:50](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1310s) And most of these large models are [21:52](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1312s) what's called a mixture of experts, [21:54](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1314s) meaning each token only uses a small [21:56](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1316s) fraction of the total weights. DeepSeek [21:59](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1319s) is 552 billion parameters, but only [22:01](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1321s) about 16 billion are active on each [22:03](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1323s) token. So, a host does 16 billion worth [22:06](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1326s) of work per token, while you, buying [22:08](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1328s) hardware, have to store all 552 billion. [22:12](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1332s) The rental market prices the work, and [22:14](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1334s) the hardware market prices the storage. [22:17](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1337s) If we think about running AI in terms of [22:19](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1339s) total cost over time, this reframes [22:21](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1341s) everything. For most tasks, the [22:24](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1344s) benchmark suite estimates a task costs [22:26](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1346s) about $3.53 [22:28](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1348s) at Quen 3.827B's rental price versus [22:32](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1352s) about $0.60 at Deep Six's price. So, the [22:34](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1354s) model you'd buy a $1,300 machine to run [22:38](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1358s) costs about six times more per task on [22:40](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1360s) rental than the model you'd spend $9,500 [22:44](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1364s) on. There's one obvious exception, and [22:46](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1366s) that's Kimmy K 3.1 Pro. It has 104 [22:50](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1370s) billion active parameters, which is a [22:52](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1372s) lot of work, and it lists at $10 per [22:55](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1375s) million output tokens. Eight DGX Sparks [22:58](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1378s) at $55,600 [23:00](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1380s) would let you generate about 5.6 billion [23:03](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1383s) tokens before the math breaks even. And [23:05](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1385s) at the 2.2 tokens per second those [23:07](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1387s) Sparks managed in the one published [23:09](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1389s) test, writing that many tokens would [23:12](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1392s) take approximately 80 years. So, for [23:15](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1395s) most people the rental market is smarter [23:17](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1397s) than the hardware market. That's why the [23:19](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1399s) advice at the end of this video is going [23:21](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1401s) to be to rent the model first, and then [23:23](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1403s) decide if you need to own the hardware. [23:25](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1405s) Now, before I get to the final [23:27](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1407s) recommendations, I want to talk about [23:29](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1409s) the ceiling that nobody advertises, [23:32](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1412s) which is speed. Kimmy K 3.1 Pro has 104 [23:36](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1416s) billion active parameters stored at [23:38](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1418s) roughly four bits, so each token it [23:40](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1420s) generates requires reading about 52 GB [23:43](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1423s) of weights from memory. To figure out [23:45](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1425s) how fast a machine can run it, we divide [23:47](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1427s) the machine's memory bandwidth by 52. A [23:50](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1430s) DGX Spark moves about 273 [23:53](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1433s) GB per second, which caps it at roughly [23:55](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1435s) five tokens per second in theory before [23:58](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1438s) networking overhead. In practice, the [24:00](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1440s) one published test of Kimmy K3 on home [24:03](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1443s) hardware spread the model across eight [24:05](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1445s) DGX Sparks, which is $55,600 [24:08](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1448s) worth of hardware, and got 2.2 tokens [24:11](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1451s) per second, which is about 14 times [24:13](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1453s) slower than the Kimmy API, and that API [24:15](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1455s) delivers a median of 31 tokens per [24:18](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1458s) second [music] and costs $10 per million [24:20](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1460s) tokens. There are software tricks that [24:22](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1462s) help. A newer engine using speculative [24:25](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1465s) decoding, which drafts ahead and [24:26](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1466s) verifies in parallel, pushed one mixed [24:29](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1469s) part of hardware from 18 tokens per [24:31](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1471s) second to between 44 and 70. So, better [24:34](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1474s) software really matters at this tier. [24:36](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1476s) But, there's a floor that hardware alone [24:38](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1478s) can push through on a home setup. At 2.2 [24:41](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1481s) tokens per second, a 300 token answer [24:44](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1484s) takes about 2 and 1/2 minutes, which is [24:46](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1486s) not a typing speed you build a workflow [24:47](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1487s) around. And, there's also the SSD route, [24:50](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1490s) which is running a multi-trillion [24:52](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1492s) parameter model off a solid state drive [24:54](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1494s) instead of RAM, and it works. With one [24:56](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1496s) published result listing 9.4 seconds per [24:59](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1499s) token. A 300-word answer at that speed [25:02](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1502s) is 47 minutes, which is technically [25:04](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1504s) running the model in the same way that [25:06](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1506s) jogging is technically running a [25:08](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1508s) marathon. [25:09](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1509s) So, after all of that, who actually [25:11](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1511s) makes sense at the terabyte realm? There [25:13](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1513s) are three real cases. The first is [25:16](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1516s) compliance. If you work in healthcare, [25:18](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1518s) law, finance, or anywhere else where [25:21](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1521s) data legally cannot leave your [25:23](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1523s) infrastructure, the rental column [25:24](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1524s) doesn't exist for you. You're not [25:26](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1526s) comparing local hardware to API pricing, [25:29](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1529s) you're comparing local hardware to [25:31](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1531s) nothing. And, in that case, the best [25:33](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1533s) open model in the world running at 2.2 [25:35](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1535s) [music] tokens per second on a local [25:37](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1537s) machine is still better than a frontier [25:39](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1539s) model you're not allowed to use. [25:41](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1541s) >> [music] [25:41](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1541s) >> The second is scale. [25:43](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1543s) Every price per token calculation I [25:45](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1545s) mentioned earlier assumes one person in [25:47](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1547s) one conversation on one [music] stream. [25:50](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1550s) And, a box that runs 10 simultaneous [25:52](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1552s) agent sessions divides its purchase [25:54](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1554s) price by 10. If you're building a system [25:56](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1556s) that runs AI continuously [music] at [25:58](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1558s) volume, the math shifts dramatically in [26:01](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1561s) favor of owning. The third is control. [26:03](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1563s) If you want the weights themselves, [26:05](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1565s) whether that's to fine-tune the model, [26:07](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1567s) to study it, [music] or to make sure [26:08](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1568s) nobody can retire it out from under you, [26:11](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1571s) a terabyte is the smallest rung that [26:13](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1573s) holds the best open model in the world [26:15](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1575s) starting November 4th. And, that's a [26:17](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1577s) different kind of value than benchmark [26:18](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1578s) scores capture. And, for everyone else, [26:21](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1581s) meaning the person who wants good AI and [26:23](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1583s) wants it fast, but doesn't have a [26:24](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1584s) specific reason to own hardware, the [26:27](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1587s) answer is to rent first. Now, let me end [26:30](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1590s) by looking at the latter one more time [26:32](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1592s) with the doublings that bought nothing [26:33](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1593s) stripped out. The first rung that [26:35](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1595s) matters is 32 GB, which is the 27B model [26:39](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1599s) with all its bits and enough context [26:40](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1600s) [music] for agent work on a $1,299 [26:44](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1604s) Mac mini. And it's the rung most people [26:46](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1606s) should start on. If you're at 16, the [26:48](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1608s) upgrade is real, and if [music] you're [26:50](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1610s) at eight, it's an easy yes. The second [26:52](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1612s) rung that matters is 128 GB, which is [26:55](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1615s) the first real step in intelligence from [26:58](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1618s) 20.1 to somewhere around 27 or 28 on the [27:01](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1621s) benchmark. If you're on a Mac, you need [27:04](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1624s) one terminal command to actually get [27:06](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1626s) there, and without it, you're paying [27:07](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1627s) $5,000 for a slightly fancier version of [27:10](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1630s) the last rung. The third rung that [27:12](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1632s) matters is 256 GB, where DeepSeek [27:16](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1636s) arrives with its 32.5. And again, on a [27:19](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1639s) Mac, there's a terminal setting between [27:21](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1641s) you and the actual model, so it's the [27:23](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1643s) same box with two very different [27:25](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1645s) experiences depending on whether you [27:26](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1646s) know the setting exists. Then there's [27:28](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1648s) the flat zone, where 512 GB is the same [27:31](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1651s) 32.5 [music] kept at four bits instead [27:34](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1654s) of three, which is a cleaner experience, [27:36](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1656s) but not a smarter one. If you want it, [27:38](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1658s) buy two 256 because it's cheaper. And [27:41](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1661s) then, from November 4th, assuming the [27:43](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1663s) weights land as promised, a terabyte [27:45](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1665s) buys 46.0, the best open model in the [27:48](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1668s) world at two-bit precision. Nobody has [27:51](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1671s) measured how much of that score survives [27:52](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1672s) compression yet, and the published [27:54](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1674s) hardware test runs 14 times slower than [27:56](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1676s) the API. But the model exists, and the [27:59](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1679s) gap is real, so the rung is real, too. [28:01](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1681s) So one doubling on this entire ladder, [28:04](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1684s) 32 to 128, gets you a [music] [28:06](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1686s) meaningfully better AI. Two doublings [28:09](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1689s) bring you to the DeepSeek tier, and one [28:11](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1691s) more at the very top gets you to a score [28:13](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1693s) nobody on home hardware has reached [28:15](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1695s) before. And everything else, which is [28:17](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1697s) 64, 512, and the terabyte before this [28:20](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1700s) week, bought the same model better [28:23](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1703s) preserved. So before you buy any of [28:25](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1705s) this, rent first by going to OpenRouter [28:28](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1708s) Artificial Analysis leaderboard or the [28:30](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1710s) ModelZone API, and spend [music] a week [28:32](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1712s) actually using the model you're thinking [28:34](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1714s) of buying hardware to run. For the 128 [28:37](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1717s) gig tier, that's a fraction of a cent [28:39](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1719s) per task, [28:40](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1720s) >> [music] [28:40](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1720s) >> and for Deep Seek, it's even less. A [28:42](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1722s) week of real usage will tell you more [28:43](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1723s) about whether you need that model than [28:45](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1725s) any benchmark number I could quote you. [28:47](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1727s) Then, buy the smallest rung that fits [28:49](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1729s) the model you actually need, which isn't [28:51](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1731s) the rung that makes the best story or [28:53](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1733s) the rung that's future-proof. It's just [28:56](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1736s) the rung that fits because the ladder [28:58](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1738s) moves under you. [music] And the week [28:59](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1739s) before this video went up, it moved at [29:01](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1741s) the top by more than it had in months. [29:04](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1744s) If you found this useful, subscribe [29:06](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1746s) >> [music] [29:06](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1746s) >> because I try to do the math that the [29:07](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1747s) tier videos leave out. And leave a [29:09](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1749s) comment with what setup you're actually [29:11](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1751s) running because every time I do one of [29:12](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1752s) these videos, I learn something new from [29:14](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1754s) the people in the comments who actually [29:16](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1756s) have the hardware sitting on their desk. [29:18](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1758s) [music] [29:18](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1758s) All right, that's the ladder, so let's [29:20](https://www.youtube.com/watch?v=kTmUbRVsamo&t=1760s) get to