Homelabber’s second-gen Optane build uses two RTX 3090s to run 2.8-trillion-parameter Kimi AI model at 3.11 tokens per second

2 hours ago 3
Open white PC case with a large CPU cooler, two GeForce RTX graphics cards and two black blower fans aimed at a row of memory modules (Image credit: u/IndividualSurvival on Reddit)

Follow us

Add us as a preferred source on Google

A homelabber detailed improvements to the second-gen Optane AI build in a follow-up post on Reddit, showing the addition of two RTX 3090s to run the 2.8-trillion-parameter Moonshot Kimi K3 model at 3.11 tokens per second. Transferring almost half the model onto a RAM drive added a significant 41% to generation speed and 26% to prompt evaluation. The extra fans are still needed, though, as the persistent Optane memory gets very hot.

Follow-up: Kimi K3 Q2_K on second-gen Optane + 2× RTX 3090 — 3.11 t/s generation, 16.8 t/s prompt eval from r/homelab

Get Tom's Hardware's best news and in-depth reviews, straight to your inbox.

Shane Downing is a Contributing Writer for Tom’s Hardware, covering consumer storage, PC hardware, and AI.

Read Entire Article