I worked with a physicist who wrote code that was so unreadable, it actually made me laugh. He would often include his initials in variable names, even though he was pretty much the only person working in the code base. His functions usually included a flags argument, which was a list of (usually undocumented) integers that you could pass in to change the behavior of the function. For example, one time one of his functions wasn’t giving the expected output, so I asked him and he replied “oh did you put 32 in the flags list?” Like he just didn’t understand that you shouldn’t need to read the entire contents of a function in order to understand how to use it.
Inb4 “well why didn’t you help him?” he was in his 70s and vehemently refused any advice.
to expand your point, the sole job of an LLM is to, when given a sequence of words (e.g. half a sentence), predict what the next several words should be. the model has no concept of what English words mean, so instead it makes this prediction based on statistics that were derived from basically reading through hundreds of thousands of English sentences
TL;DR LLMs don’t understand languages, they’ve just memorized statistics about them
I can smell the fedora through this post