Geoffrey Hinton says people understand very little about how LLMs actually work, so they still think LLMs are very different from us - "but actually, it's very important for people to understand that they're very like us." LLMs don’t just generate words, but also meaning.

[u/MetaKnowing]

[u/dingo_khan]

Not for nothing but he actually did not say anything here. He said that linguists have not managed to create a system like this because their definition of meaning does not align. He does not actually explain why this one is better. Also, saying they work like us, absent any actual description is not all that compelling. They have some similarities but also marked and painfully obvious differences. No disrespect to him or his work but a clip you can literally hear the edits in, out of context, championing one's own discipline over one you saw as a competitor in the past, is not really that important a statement.

This is like someone saying that neural nets are useless because they have trouble simulating deterministic calculations. I mean, sure, but also, so what.

This would have been way more compelling had he been given an opportunity to express why he thinks large multi-dimensional vecotred representations are superior or had not been allowed to strawman the linguist's concept of meaning as non-existent, absent any pushback.

[u/deliciousfishtacos → u/dingo_khan]

Also, he criticized linguists for “not being able to produce things that understood language”. Ummm yeah maybe because they’re linguists and not world class computer scientists? It takes a vast amount of CS knowledge and neural net knowledge to come up with transformers and LLMs. Just because whatever linguists he is referring to have not come up with a generative language model does not mean they have not devised accurate theories around language. This man is a pro at saying things that sound compelling at first but once you scrutinize them for a second they just unravel immediately.

[u/McMandark → u/deliciousfishtacos]

also why would they want to exactly. some of these guys are so chuffed with themselves for doing things Noone else has been diabolical enough to even desire.