Response to Goddu et al.: new ways of characterizing and acquiring knowledge

Published on September 20, 2024

In a recent opinion in TiCS [1], we asked: ‘What do large language models (LLMs) know?’. We answered by granting LLMs instrumental knowledge: that is, knowledge gained through using the instrument of next-word generation. We then explored how this type of instrumental knowledge could be related to the more ordinary kind of worldly knowledge exhibited by humans, including a review of computational literature where training on next-word prediction spontaneously recovered structure-preserving mappings of the training domain.

Read Full Article (External Site)