# I Gave Local AI and the Cloud the Exact Same Job
Canonical: https://social-archive.org/jack_bauer111/KsCbF0a9NO
Original URL: https://www.youtube.com/watch?v=ujs0_cpAnaw
Author: Alex Ziskind
Platform: youtube
Share mode: full
## Content
I used my $60,000 512GB Mac Studio cluster running Kimi K3 to see if local AI can beat a $10/month cloud coding agent, and the result was way not what I expected. 🔹 Try out Supercomputer - https://supercomputer.abacus.ai/ltf 🛒 Gear Links 🛒 🪛🪛Highly rated precision driver kit: https://amzn.to/4fkMVfg 💻☕ Favorite 15" display with magnet: https://amzn.to/3zD1DhQ 🎧⚡ Great 40Gbps T4 enclosure: https://amzn.to/3JNwBGW 🛠️🚀 My nvme ssd: https://amzn.to/3YLEySo 📦🎮 My gear: https://www.amazon.com/shop/alexziskind 🎥 Related Videos 🎥 🧱🖥️ Mac Studio Cluster - https://youtu.be/25xVqvL5j4g 🧳🧰 Mini PC portable setup - https://youtu.be/4RYmsrarOSw 🍎💻 Dev setup on Mac - https://youtu.be/KiKUN4i1SeU 💸🧠 Cheap mini runs a 70B LLM 🤯 - https://youtu.be/xyKEQjUzfAk 🧪🔥 RAM torture test on Mac - https://youtu.be/l3zIwPgan7M 🍏⚡ FREE Local LLMs on Apple Silicon | FAST! - https://youtu.be/bp2eev21Qfo 🧠📉 REALITY vs Apple’s Memory Claims | vs RTX4090m - https://youtu.be/fdvzQAWXU7A 🧬🐍 Set up Conda - https://youtu.be/2Acht_5_HTo ⚡💥 Thunderbolt 5 BREAKS Apple’s Upcharge - https://youtu.be/nHqrvxcRc7o 🧠🚀 INSANE Machine Learning on Neural Engine - https://youtu.be/Y2FOUg_jo7k 🧱🖥️ Mac Mini Cluster - https://youtu.be/GBR6pHZ68Ho * 🛠️ Developer productivity Playlist - https://www.youtube.com/playlist?list=PLPwbI_iIX3aQCRdFGM7j4TY_7STfv2aXX 🔗 AI for Coding Playlist: 📚 - https://www.youtube.com/playlist?list=PLPwbI_iIX3aSlUmRtYPfbQHt4n0YaX0qw My Silicon Exchange site prompt: https://gist.github.com/alexziskind1/fe55f03892f3fe1d6d8cf6065c631bb8 — — — — — — — — — ❤️ SUBSCRIBE TO MY YOUTUBE CHANNEL 📺 Click here to subscribe: https://www.youtube.com/@AZisk?sub_confirmation=1 — — — — — — — — — Join this channel to get access to perks: https://www.youtube.com/channel/UCajiMK_CY9icRhLepS8_3ug/join — — — — — — — — — 📱 ALEX on X: https://x.com/digitalix #macstudio #gdxspark #strixhalo --- ## Transcript [0:00](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=0s) This is a 2.8 bill, [0:02](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=2s) sorry, this is a 2.8 trillion parameter [0:05](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=5s) model and it looks like it's going to be [0:06](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=6s) the biggest one for a while. Kimmy K3, [0:09](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=9s) 1.56 terabytes on disk. It doesn't fit [0:12](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=12s) on a single Mac Studio, not even in Q4, [0:14](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=14s) not even in Q2. That's the quantization [0:16](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=16s) I'm talking about when you shrink the [0:17](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=17s) model from its original weights. So, I [0:20](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=20s) wired up four Mac Studios together, each [0:23](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=23s) with 512 GB for a total of 2 TB of [0:27](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=27s) unified memory. 2 TB minus 1.56 Yeah, [0:30](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=30s) we're good. And I wanted to see if it [0:31](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=31s) can beat the new Abacus AI [0:34](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=34s) supercomputer. I'm going to have Kimmy [0:36](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=36s) K3 build a web app, a front-end web app, [0:39](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=39s) and I'm going to have supercomputer do [0:40](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=40s) the same. It's local hardware versus [0:42](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=42s) cloud death match, round two. Let's go. [0:45](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=45s) So, last time I did this it was one Mac [0:47](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=47s) Studio, 512 gigs against Abacus deep [0:50](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=50s) agent. And a lot of you mentioned in the [0:51](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=51s) comments, "Alex, that's not a real local [0:54](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=54s) setup." Fair enough. So, this time I [0:56](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=56s) brought four Mac top and yeah, it's [0:59](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=59s) running on it right now charted across [1:01](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=61s) all four Mac Studios connected with [1:03](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=63s) RDMA. Mac Studio 1, 208 GB used, almost [1:07](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=67s) 200 used on Mac Studio 2, 177 [1:10](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=70s) on Mac 3, and 215 on Mac 4. For context, [1:14](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=74s) Kimmy K2 was 1 trillion parameters and [1:17](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=77s) that already felt pretty ridiculous. [1:19](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=79s) Kimmy K3's almost three times that. And [1:21](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=81s) the more parameters you have, the more [1:23](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=83s) the model can actually do. It's smarter. [1:25](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=85s) That's the whole trade-off. A 1 billion [1:27](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=87s) parameter model gives you gibberish. No [1:29](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=89s) that's Now, even though 1.56 [1:33](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=93s) TB in the original 8-bit precision, I'm [1:35](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=95s) still running a smaller quantization, [1:37](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=97s) but it's unpruned. I'm running all 896 [1:41](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=101s) experts. That's a total of 817 GB of [1:44](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=104s) weights sitting on disk. Now, you might [1:46](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=106s) think, "Oh, each one of those is 512, [1:49](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=109s) you can fit that on two, right?" Well, [1:50](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=110s) not exactly because I'm giving it a [1:52](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=112s) large amount of context, you're going to [1:53](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=113s) need a lot more space. And you could [1:56](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=116s) say, "Well, why not use three then? [1:58](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=118s) Because tensor parallelism splits the [2:00](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=120s) model dimensions across nodes. And those [2:03](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=123s) dimensions have to be divided evenly. [2:04](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=124s) So, it's either one, two, or four, or [2:07](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=127s) eight. Kimik4, maybe next year. And all [2:09](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=129s) four of these are connected through [2:10](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=130s) Thunderbolt 5 mesh on the back here. Six [2:13](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=133s) cables total connecting every machine to [2:15](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=135s) every other machine. I made a whole [2:16](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=136s) video describing how I did that and how [2:19](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=139s) I ran it with MLX distributed. You can [2:21](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=141s) check that out. I'll link to it down [2:22](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=142s) below. All right. [2:24](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=144s) Let's see your pretty face. Now, let's [2:26](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=146s) meet the other contender. This is Abacus [2:28](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=148s) AI, and they have a new tool called [2:30](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=150s) Supercomputer. You've probably already [2:32](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=152s) seen that they sponsored some of my [2:33](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=153s) content in the past, and they're [2:34](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=154s) sponsoring this video, too. But, I keep [2:36](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=156s) coming back to it on my own time because [2:37](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=157s) essentially it gives me one login to be [2:40](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=160s) able to do and use every frontier model [2:43](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=163s) out there on day one. There's over 100 [2:45](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=165s) models here, and route LLM basically [2:47](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=167s) routes you to the right one based on [2:49](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=169s) your context. So, there's GPT 5.6 Soul. [2:52](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=172s) That's pretty new as of this video. [2:54](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=174s) Llama 2 5, Cloud Fable 5, Grok 4.5. You [2:58](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=178s) know, they got image generation and [2:59](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=179s) things like that. They also have Kimik3 [3:01](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=181s) here, too. DeepSeek V4 Flash and Pro GLM [3:04](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=184s) 5.2. Open-source models, close-source [3:06](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=186s) models, everything. Kind of cool cuz [3:08](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=188s) you're not locked into a single [3:09](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=189s) ecosystem. Now, what's different about [3:11](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=191s) Supercomputer is that it's actually a [3:13](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=193s) machine, a virtual machine. It's not [3:15](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=195s) just a chat window. It's a cloud box [3:18](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=198s) that's always on that's also happens to [3:20](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=200s) be integrated into their agent. You [3:22](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=202s) don't wait for a cold start. You don't [3:24](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=204s) wait for a spin-up. You never see [3:25](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=205s) container provisioning because it's [3:28](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=208s) always on. Scheduled tasks, databases, [3:30](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=210s) storage, GitHub integration, SSH. And [3:33](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=213s) also, they got Hermes agent now. So, [3:35](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=215s) here I've got Hermes running, and I can [3:37](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=217s) change models to whatever I want. So, [3:39](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=219s) I've selected Kimik3. I can select [3:42](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=222s) whatever other model I want to talk to [3:44](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=224s) directly, like GPT 5 Luna. [3:47](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=227s) Say hi to Luna. So, you got Hermes. You [3:49](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=229s) don't need to install it locally. It's [3:51](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=231s) always on and running over there. I'll [3:52](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=232s) talk about the pricing in a bit. So, the [3:55](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=235s) fight is pretty simple. They're going to [3:56](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=236s) get the same prompt, both sides, and I [3:58](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=238s) got the prompt as a gist. You can check [4:01](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=241s) it out. I'll link to it down below. It's [4:02](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=242s) a pretty extensive long prompt, but it's [4:04](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=244s) specifically front end for a silicon [4:07](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=247s) compute exchange front end. I want to [4:09](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=249s) make sure we're using TypeScript with [4:10](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=250s) the latest and greatest front end tech. [4:12](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=252s) We're using mock data, not real data, [4:14](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=254s) but this can easily be swapped out [4:15](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=255s) later. Dark mode, microinteractions, [4:18](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=258s) accessibility, we should have a pretty [4:21](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=261s) decent-looking site in the end, [4:22](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=262s) hopefully. It's going to have some unit [4:24](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=264s) testing, and it needs to check its own [4:26](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=266s) work. So, this is the agentic part of [4:28](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=268s) it. It's going to be running all these [4:30](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=270s) commands to be able to verify and [4:32](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=272s) validate that it's working. Okay, let's [4:34](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=274s) get it. Raw copy. I'm going to head over [4:37](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=277s) to supercomputer and just paste this in [4:40](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=280s) into the agent. Now, here it's going to [4:41](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=281s) be set to auto, but you know what? I'm [4:43](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=283s) going to go with Opus 5 high and GPT 5.6 [4:48](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=288s) soul. Oh, yeah, you have access to the [4:50](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=290s) full virtual machine here on the right, [4:52](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=292s) too. You got the files that it's going [4:53](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=293s) to generate, access [music] directly to [4:55](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=295s) the terminal, and access to the desktop. [4:57](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=297s) Now, I don't want to give it a leg up, [4:59](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=299s) so I'm going to go and start this off in [5:02](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=302s) my Mac Studio cluster, too. And the way [5:03](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=303s) I set this up locally is I just have [5:05](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=305s) Open Code, which is the local agent, [5:07](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=307s) pretty popular, but you can point it to [5:10](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=310s) any local model that you want. And I've [5:11](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=311s) pointed it to the 4x Mac Studio cluster [5:14](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=314s) running Kimiko 3. I actually made it a [5:16](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=316s) separate video on how to set this up for [5:18](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=318s) members of the channel. By the way, [5:20](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=320s) thank you to the members for supporting [5:22](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=322s) the channel. Let's paste in that prompt, [5:24](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=324s) and boom. [5:27](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=327s) There we go. [music] It's doing stuff, I [5:29](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=329s) hope. How do I know? Here's all four of [5:31](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=331s) them, and the GPU usage is pegging 100% [5:34](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=334s) on all four of these machines. Do I hear [5:36](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=336s) it? [5:39](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=339s) No, I don't hear it. Do I feel it? It's [5:41](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=341s) warm. And if I put my ear right up into [5:43](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=343s) it, then I'll hear a little little [5:44](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=344s) twinkle twinkle little star kind of [5:46](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=346s) sound in there. Maybe the fans will kick [5:48](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=348s) up later, but it's working. Do we have [5:50](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=350s) anything built yet? No, it's thinking. [5:52](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=352s) Where is my output? All right. All [5:55](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=355s) right, it's doing it. I'm going to start [5:57](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=357s) the other one. Boom, I'll build that for [5:59](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=359s) you right now. This will basically do [6:01](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=361s) the same thing. It's building it. Let's [6:02](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=362s) pop this open on the right so we can see [6:04](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=364s) what's happening. By the way, this is [6:06](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=366s) the first time I'm actually running that [6:07](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=367s) prompt. I don't know what's going to [6:08](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=368s) happen. [6:09](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=369s) Hopefully it works. It says if you [6:11](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=371s) connect your GitHub, I can work with [6:13](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=373s) your repositories directly. That's [6:14](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=374s) pretty cool. Cloning, pushing, commits, [6:17](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=377s) and pull requests. Want me to set that [6:18](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=378s) up? Go ahead. So now Abacus AI agent [6:21](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=381s) takes over and we've seen it at work [6:23](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=383s) before, but the agent is now well [6:24](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=384s) integrated into the supercomputer [6:26](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=386s) concept that's always on and it's going [6:29](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=389s) to be deploying to this VM. Building a [6:31](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=391s) production-ready Next.js web app. Now [6:33](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=393s) compared to some of the other builds I [6:35](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=395s) I've done here on the channel with [6:37](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=397s) Nvidia and DGX Sparks, the prefill or [6:40](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=400s) the prompt processing stage of inference [6:42](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=402s) is something that Max are not as good at [6:46](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=406s) as, for example, Nvidia GPUs. Macs are [6:49](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=409s) really good at generating once they've [6:52](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=412s) processed because they have extremely [6:54](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=414s) good high memory bandwidth. And with [6:56](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=416s) Kimiko 3, I measured about a 238 tokens [6:59](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=419s) per second chewing through the prompt, [7:01](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=421s) which is not super fast. Oh. [7:04](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=424s) Now I can hear it. There's work to be [7:06](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=426s) had. Generation happens at about 14.7 [7:10](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=430s) tokens per second with Kimiko 3 here. [7:12](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=432s) It's not going to be fast, folks, but it [7:15](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=435s) is a big model. What's happening on the [7:16](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=436s) Abacus side? Well, we already have some [7:18](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=438s) HTML generated. Now while it's doing [7:20](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=440s) that, we can take a little peek under [7:23](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=443s) the hood and see what's happening. So [7:25](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=445s) I'm going to go in here and let's do uh [7:28](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=448s) this is just Ubuntu over here. We got [7:29](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=449s) skills. This is just a generic skills [7:33](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=453s) repo. It doesn't have any custom skills [7:35](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=455s) installed. So there is Silicon Exchange. [7:37](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=457s) Let's go in there. That's our app. [7:38](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=458s) Next.js space. And yeah, there's our [7:41](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=461s) scripts and Tailwind config, TypeScript [7:43](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=463s) configuration, Next.js config. I wonder [7:46](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=466s) what's in that style guide. I wonder if [7:48](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=468s) this is something that it generated just [7:50](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=470s) now based on my prompt or [7:53](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=473s) just a generic one for Next.js projects. [7:56](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=476s) Anybody know? [8:01](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=481s) >> [snorts] [8:02](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=482s) >> Please, sir, I want a token. Now, [8:04](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=484s) nobody's going to believe me that this [8:06](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=486s) Kimmy actually works. I swear, I tried [8:08](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=488s) it. Didn't try this prompt, but I tried [8:11](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=491s) using it and it worked fine. Maybe this [8:13](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=493s) prompt is just too hard. Now, let's take [8:15](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=495s) that 14 tokens per second figure. We can [8:17](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=497s) use something called Abacus AI Desktop, [8:20](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=500s) which is their desktop companion app. [8:22](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=502s) It's got chat, co-work, and code. And [8:25](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=505s) this is basically like a local agent. [8:27](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=507s) They have a CLI, too, but they have a [8:29](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=509s) graphical interface. It's kind of like [8:30](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=510s) Codex. The difference is you can pick [8:33](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=513s) the model that you want right here from [8:34](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=514s) this drop-down. Fable 5 is there, Opus [8:36](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=516s) 5, GPT 5.6, Soul, Terra, Luna, Grok is [8:40](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=520s) there, and Kimmy K3 is there, too. So, [8:42](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=522s) let's see what we get here. Of course, [8:44](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=524s) I'm using code here, so I'm going to [8:46](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=526s) need to select a workspace. Write me a [8:48](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=528s) simple JavaScript Node application for [8:51](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=531s) adding two numbers. Boom. Kimmy K3. Now, [8:54](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=534s) this is not using my Kimmy K3. This is [8:56](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=536s) using the Kimmy K3 that Abacus uses. But [8:59](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=539s) it's writing a local application. So, we [9:01](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=541s) need to accept a few things here. Allow [9:04](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=544s) it. Let's do a bypass, and there it [9:06](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=546s) goes. Now, that is pretty fast. I think [9:09](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=549s) it's done. Here you can access the VS [9:11](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=551s) Code extension, browser extension. So, [9:13](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=553s) this desktop app has a lot of extra [9:15](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=555s) abilities that it adds. Let's see. Node, [9:18](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=558s) add five and four. Nine. Is that right? [9:22](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=562s) Think so. What's happening over here [9:23](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=563s) with our Kimmy? Oh, no token yet. [9:27](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=567s) Well, it's been a while, so I had to [9:29](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=569s) check what's going on, and [9:31](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=571s) it wasn't printing out any tokens at [9:33](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=573s) all. So, I'm going back into it to see [9:35](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=575s) if I can even get anything out of it. [9:38](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=578s) I'm going to just say hi. Does look like [9:39](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=579s) it's working. The GPUs are at 100%, so [9:43](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=583s) yeah. Hi, how can I help you today? It [9:45](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=585s) worked. Maybe I just wasn't patient [9:48](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=588s) enough for such a large prompt. Let's do [9:50](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=590s) this. Create a simple web app with a [9:52](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=592s) text box and a button. Well, look at [9:54](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=594s) that. [9:55](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=595s) It just printed out index.html for me. [9:58](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=598s) Only took about 3 minutes to 147 lines. [10:02](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=602s) It's a beautiful, beautiful app. Wow. [10:06](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=606s) Test, submit, and it works. Still [10:09](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=609s) printing out. It's not done yet. I've [10:10](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=610s) created a simple web app for you. It's [10:12](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=612s) It's doing the explainer. Thank you. [10:14](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=614s) Thank you for that, Kimmy. I mean, you [10:15](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=615s) did a fantastic job. I just wish you did [10:18](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=618s) it faster. [10:20](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=620s) Okay, after all that, I couldn't just [10:23](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=623s) leave it like that. I couldn't leave you [10:24](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=624s) hanging. So, I stopped recording, had [10:26](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=626s) some coffee, went back and gave Kimmy K3 [10:29](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=629s) the full prompt again. But, this time I [10:31](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=631s) didn't sit there and watch it because, [10:33](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=633s) you know, watching something It's kind [10:35](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=635s) of like watching grass grow. You don't [10:37](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=637s) think anything's happening, but [10:38](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=638s) something is. I had to walk away and [10:41](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=641s) have more coffee. Guess what happened 4 [10:42](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=642s) hours later? We have a full app. Here it [10:45](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=645s) is. [10:46](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=646s) And it looks pretty good. This was done [10:49](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=649s) by Kimmy on these machines. There's the [10:52](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=652s) browse. We have live filtering, reset [10:54](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=654s) all filters, inspect, compare. We have [10:57](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=657s) charts. We have scheduling. This is a [11:00](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=660s) beautiful interface. And it can do light [11:03](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=663s) mode. It can do dark mode. Sorry about [11:05](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=665s) that. What do you think? I think it did [11:06](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=666s) a fantastic job here. So, the question, [11:08](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=668s) can you do this locally? The answer is [11:11](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=671s) yes. Yes, you can. Let's take a look at [11:13](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=673s) what Abacus came up with. It looks like [11:14](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=674s) it's actually finished. So, let's pop [11:17](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=677s) open our VM here. Still looking at [11:18](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=678s) exchange, and then what's the name of [11:20](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=680s) that? Next.js app. And we're going to do [11:22](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=682s) npm run dev. Okay, it built and started [11:26](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=686s) it at this URL. Let's take a look at [11:29](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=689s) that. Okay. Wow. Can't get too excited [11:32](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=692s) cuz it is a sponsored video. Wow. Oh my [11:36](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=696s) gosh. [11:37](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=697s) >> [laughter] [11:38](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=698s) >> I mean, it is pretty cool. It's very [11:40](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=700s) cool. It's a beautiful application. The [11:42](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=702s) design is quite something. The dashboard [11:45](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=705s) is beautiful. You got filters here on [11:47](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=707s) the left doing live filtering with [11:50](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=710s) animation. This is from a single prompt. [11:53](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=713s) I mean, this is not hooked up to a back [11:54](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=714s) end, but it wouldn't be that difficult [11:56](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=716s) to take it and extend it there. Your [11:58](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=718s) reservations, compare boards, browse [12:01](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=721s) inventory goes back to that page. Wait, [12:03](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=723s) what does this do over here? Let's do [12:04](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=724s) Apple M3 Ultra available. Simple web [12:08](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=728s) app. Look, I'm sure the capability is [12:10](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=730s) there of this model [12:13](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=733s) with the right hardware. I'm capable to [12:15](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=735s) run it. It's just not very fast. And I [12:17](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=737s) like to code. And the way I use the [12:19](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=739s) agents is to do these one-off tools that [12:22](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=742s) I use. My GitHub lately has been a bunch [12:26](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=746s) of different tools that I implemented [12:29](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=749s) using the help of agents. These are [12:31](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=751s) things that I want to knock out quickly [12:33](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=753s) and get working. And for that kind of [12:35](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=755s) stuff, Abacus supercomputer is perfect. [12:37](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=757s) Same prompt, same app, both finished it. [12:40](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=760s) 15 minutes against 4 hours. Now, each [12:43](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=763s) one of those Mac Studios was about [12:45](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=765s) $16,000, but now I think they're not for [12:48](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=768s) sale anymore, so it's a lot more. And I [12:50](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=770s) don't know how much the new ones are [12:51](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=771s) going to cost, but you get the idea. [12:52](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=772s) It's expensive hardware. You buy it one [12:55](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=775s) time, then it's yours, and then you do [12:57](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=777s) whatever you want with it. Abacus starts [12:59](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=779s) at 10 bucks a month. Actually, $7 for [13:03](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=783s) the first month now. You could run that [13:04](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=784s) subscription for over 300 years before [13:06](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=786s) you spend that kind of money. And you [13:08](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=788s) can use their web UI, you can use the [13:11](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=791s) desktop application that I showed you. [13:13](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=793s) You can use their APIs. You're also not [13:15](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=795s) stuck at one price because there's [13:16](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=796s) custom routers that let you push the [13:19](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=799s) easy work to cheaper models and save the [13:21](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=801s) expensive ones for the harder stuff. So, [13:22](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=802s) the bottom line is, and it's not what [13:25](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=805s) sponsors usually want to hear, Yes, you [13:27](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=807s) can do all the stuff locally, and that's [13:30](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=810s) pretty remarkable. That wasn't possible [13:32](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=812s) just a year ago. But, if you want high [13:33](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=813s) quality, and you want it done fast, and [13:35](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=815s) you want to be able to host it all in [13:37](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=817s) one shot, Abacus's cloud solution wins [13:39](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=819s) this one. If your data can't leave the [13:41](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=821s) building, or you just love running this [13:43](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=823s) stuff yourself, like I do, Cluster. [13:45](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=825s) There's definitely a market for both, [13:47](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=827s) not one or the other. To see me build [13:49](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=829s) and configure that cluster and all the [13:50](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=830s) details of how I run it, watch this [13:52](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=832s) video right here. And watch this video [13:54](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=834s) when I compare a Mac Studio against [13:56](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=836s) Abacus's AI. By the way, that website is [13:59](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=839s) still up and running. [14:00](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=840s) >> [music] [14:00](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=840s) >> Thanks for watching, and I'll see you [14:01](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=841s) next time. [14:06](https://www.youtube.com/watch?v=ujs0_cpAnaw&t=846s) >> [music]