[LM] The Debate Over Understanding in AI’s Large Language Models
Date:
The article describes what LLM is, how it is trained and works, and how it works, and then shows that LLM is also brittle and can make mistakes when disturbed. The article points out that there are two opposing voices, one arguing that this is the budding of general intelligence, and the other arguing that LLM only learns the form of language rather than its meaning. The article also shows, with some examples, that shortcut learning, a phenomenon often invoked in machine learning, is also present in LLM, i.e., learning systems that rely on spurious correlations in the data.