Public shared posts

jack_bauer111 shared this post · 57m ago
Alex Ziskind

I Gave Local AI and the Cloud the Exact Same Job

I used my $60,000 512GB Mac Studio cluster running Kimi K3 to see if local AI can beat a $10/month cloud coding agent, and the result was way not what I expected.

718 1.1M
@rikachiu Alternatively, I can use an enterprise Claude Max plan with contractual data-retention terms for roughly $2,000 a year. With the available discounts, that also gives me around $1,500 in monthly API credits. Using Fable 5, I can have the same project built, deployed to my custom sandbox, and configured with OAuth and the security controls I need in under 45 minutes.

When better models arrive, I can switch to them immediately, or use their APIs as needed at a 30% discount.

It would take roughly 30 years of subscriptions to equal the cost of that Mac Studio cluster.

Sure, the cluster is something I can say I own.

I’m pro–open source, but for enterprise work, it is currently too slow and doesn’t provide meaningful savings. For personal projects, though, it can make sense—if you have $60K+ to invest.
33 likes
@DejayClayton I'd really like to know what "enterprise" Claude Max plan you have access to for $2k a year, that also provides $18k a year in API credits. 8 likes
@rikachiu  @DejayClayton the beauty of being an NGO
@DejayClayton  @rikachiu  ok, so in other words, you’re in a situation that only applies to 0.0001% of people. Got it. 8 likes
@rikachiu  @DejayClayton  Maybe, but local AI is unlikely to ever going to match cutting-edge frontier models. One individual simply cannot compete with the scale of a full data center; especially once quantum computing and advanced world models become mainstream. It’s an unrealistic expectation and a power/compute restraint no matter how much we want it.

But, I am ready to be proven wrong.
1 like
@DejayClayton  @rikachiu I agree, but one always has to consider the cost when evaluating the value proposition of local AI vs hosted AI, and in many cases, the usage costs of hosted AI can be out of reach for some people. 1 like
@FlareBlossom  @rikachiu  what is an NGO? 1 like
@rikachiu  @FlareBlossom  A non-profit group that operates independently of any government to address social, environmental, and humanitarian issues

I rather keep the org I work for private.
1 like
@tomcanham9218  @DejayClayton  and there's something to be said for owning your own compute infrastructure, especially when cloud providers are doing so much fuckery. I'll keep going down the route I'm going down, where local/sovereign AI is the main driver, and cloud inference is an expensive add-on.

Also, everyone is still stuck in the "bigger is better" mindset. That's wrong. Sometimes bigger isn't just not better, but actively worse.
2 likes
@UnknownIllusionist  @rikachiu  the monthly plans are subsidized compared to the api rates. Chatgpt announced they are cutting $200 plan in half. Your assumption assumes the cheap cloud rates will continue to be cheap and local Ai software won't improve at the same time 3 likes
@rikachiu  @UnknownIllusionist  Sure, but you get more intelligence and cheaper per task completion. Don't forget you have to also spend on power to use anything locally. Local is still for pet personal projects, not to run an enterprise stack.

I also use local for mundane tasks that I can run 24/7, but no matter the hype, local just doesn't compare to bleeding edge frontier no matter how much people want it.
@mdzaid5925 My understanding is that smaller models with very high thinking mode performs better than bigger models on lower thinking level. 1 like
jack_bauer111 shared this post · 58m ago
Byte Sized Explainer

Every GPU Feature Explained in 9 Minutes

Explaining Every GPU Feature in just 9 Minutes — including Ray Tracing, Path Tracing, DLSS, FSR, Frame Generation, HDR, V-Sync, G-Sync, FreeSync, Anti-Aliasing, Anisotropic Filtering, Dynamic…

892 2.3M
@aryangarg0608 now i finally understand each thing what it does exactly highly recommended video great work hermanos! 2 likes
@SameerArolkarAGMsSs I already knew most of most of these but glad to know few of this which i had no idea about a short detailed video. 4 likes
jack_bauer111 shared this post · 59m ago
Kai

Don't Buy a GPU Instead Build This 32GB Free AI Server Instead

Don't buy a GPU yet. Kaggle gives you a Free GPU AI server with 32GB VRAM (two Tesla T4 cards), and every tutorial calls it the best Free AI setup for Local AI coding.

383 433.5K
@futuristicstoic Thanks for this wonderful video 🔥 1 like
@Dave5281968 Isaac Asimov coined an acronym, TANSTAAFL, in the 1980's. It's short for, "There Ain't No Such Thing As A Free Lunch."
The best way I've seen to do it, assuming your GPU budget is $0.00, and it must be local is either 1) use an old (preferably) nVidia GPU or use you iGPU. And fall back to a reasonably new (Ryzen 7 5xxx or Core I7-10xxx, or better) desktop CPU to do the work. Sounds crazy, I know, but it will get the job done. And might surprise you with how well it can do. CPU's can run bigger models, while an old GPU is going to be very size constrained. The CPU is also cheaper to run.
1 like
jack_bauer111 shared this post · 59m ago
Ray Fu

Complete Guide to building an AI Trading Bot with Claude for Beginners

🔥 Check out all my free Tech and AI guides here: https://www.raycfu.com

🔥 Check out my community: https://www.skool.com/raycfu if you want to make money and learn how to run your business with AI,…

1.3K 636.5K
@Phofia I’m an AI beginner, but I’ve started using it for research, helping me to study, and helping with my job search. I would love to start making money with AI 3 likes
@joepikka9923 I use AI almost every day. It helps me with studying, learning new topics, coding, fixing errors, building websites and apps, brainstorming ideas, and planning my projects. Instead of just giving me answers, I mostly use it like a personal assistant that helps me understand things faster and get more work done. AI has honestly become a normal part of my daily routine. 27 likes
jack_bauer111 shared this post · 1h ago
Ray Fu

Complete Guide to building Jarvis with Claude

🔥 Check out my community: https://www.skool.com/raycfu if you want to make money and learn how to run your business with AI

🔥 Check out all my free Tech and AI guides here including one on building…

160 30.0K
@shearstylerhys I want to use Jarvis to automate my content creation pipeline and make posting and researching for videos easier! 1 like
@Aryaman_Mishra i use ai to automate cold emails, write detailed prompts and generate videos,images and what not. i hit claude limit 3 times on my two main accounts and twice on other 3