“from earlier models like GPT-3, maybe to GPT-4—I don't remember when the transition happened exactly—they shot up from about 8K to 100-200K.”
Books mentioned in How GPT, Claude, and Gemini are actually trained and served – Reiner Pope
From Dwarkesh Podcast · 2026-04-29