LLM randomness exists to keep up the illusion
Quote
Deterministic, like if you have the temperature at zero and you don't introduce any randomness, it should literally give you the same output every time. But nobody does it that way because it drops the illusion.
Episode 241: RSS NutJobs (episode page, opens in a new tab) 00:07:10 (play audio from this timestamp, opens in a new tab)
Voice-matched
Summary
Dave notes that at temperature zero, with no randomness added, an LLM is deterministic and gives the same output every time. Nobody runs it that way, he says, because that would drop the illusion. The 'magic' feeling depends on the randomness, which is smoke and mirrors.