Language models only know word order, not meaning
Quote
you immediately see a thing in your head, because you know what those words mean? This thing only knows the language models only know, the text that it seen before and the arrangement of in the order of the words
Episode 168: Frog Giggin' (episode page, opens in a new tab) 00:11:01 (play audio from this timestamp, opens in a new tab)
Speaker-labeled
Summary
When a person hears 'a podcast item with an HLS alternate enclosure', they picture what it means. A language model only knows the text it has seen and the order of the words, which is why it hallucinates attributes that sound right.