“from earlier models like GPT-3, maybe to GPT-4—I don't remember when the transition happened exactly—they shot up from about 8K to 100-200K.”
How GPT, Claude, and Gemini are actually trained and served – Reiner Pope · Dwarkesh Podcast
Gpt-3
Sandra Kublik, Shubham Saboo
Where Gpt-3 was mentioned: 6 timestamped quotes from 6 episodes across 1 podcast.
Every mention
“the scaling insight arrived. Scaling laws, GPT-3, and suddenly everyone realized we should scale.”
Ilya Sutskever – We're moving from the age of scaling to the age of research · Dwarkesh Podcast
“coming out of. So GPT3, if you asked it, “are bugs real?” It would give this kind of hemming hawing answer like “oh, we can never truly tell what is real, who knows?” Because it was trained kind of,”
AI 2027: month-by-month model of intelligence explosion — Scott Alexander & Daniel Kokotajlo · Dwarkesh Podcast
“ones deployed in the API. Or after GPT-3, those were the next generation of models. At the same time there were definitely a lot of people thinking about chat. Google had some”
John Schulman (OpenAI Cofounder) — Reasoning, RLHF, & plan for 2027 AGI · Dwarkesh Podcast
“very good, it's almost GPT-3 quality, and runs on my laptop. I've grown up in this world of tinkerers and open-source folks and the more access you have,”
Nat Friedman (Github CEO) — Reading ancient scrolls, open source, & AI · Dwarkesh Podcast
“the gbt3 is like um only 175 billion parameters or”
Joseph Carlsmith - Utopia, AI, & Infinite Ethics · Dwarkesh Podcast