This 125B AI Model Runs 5x Faster on My PC Than in the Cloud (Using Strata)

We ran a 125-billion-parameter AI model on a single RTX 5090 using Strata, a free open-source engine, and it hit over 250 tokens per second, three to five times faster than Qwen 3.8 Flash in the cloud through OpenRouter. We break down how Strata splits the model across your GPU, CPU, RAM and SSD, including its expert cache, the “doorbell” trick that lets the CPU and GPU work on the same layer at once, and speculative decoding. Then we put both models through side-by-side coding, logic, creative and long-answer tests to see where the speed gap really shows.

🔗 Relevant Links

Strata: https://github.com/Niko1221/Strata

❤️ More about us

Radically better observability stack: https://betterstack.com/

Written tutorials: https://betterstack.com/community/

Example projects: https://github.com/BetterStackHQ

📱 Socials

Twitter: https://twitter.com/betterstackhq

Instagram: https://www.instagram.com/betterstackhq/

TikTok: https://www.tiktok.com/@betterstack

LinkedIn: https://www.linkedin.com/company/betterstack

📌 Chapters:

0:00 Local AI vs the cloud

0:38 What is Strata?

1:31 The model behind it

1:58 How Strata uses your GPU, RAM and SSD

2:34 The “doorbell” trick

3:04 Speculative decoding

3:29 Why it’s so fast on an RTX 5090

4:23 Setting up Strata

5:17 The Desk vs Cloud test app

5:30 Coding test

5:48 Logic puzzle test

6:12 Creative test

6:22 Long answer test

7:10 Long answer with thinking off

7:23 Cost and caveats

7:56 Final thoughts