Hard to say. Like, do “Ivan”, “Giovanni” and “John” count as the same, or different names? What about Latin “Amanda” (to be loved) vs. Japanese 愛/Ai (love)? How do we even count this?

Fully agree.

I hope that it doesn’t though. We don’t need yet another centralised platform that can and will go rogue, and is already showing signs that it wants to go rogue.

That is an enormous amount of words to say, “Elon Musk is a piece of shit.”

More like “Elon Musk is a piece of shit and he bought Twitter on a whim, because he’s bored, too rich to care, too big to fail.”

Without Einstein, I think that the discovery of time dilation would be delayed by only a few years. There were a lot of people working in theoretical physics already back then; someone else would inevitably dig through Lorentz’ and Poincaré’s papers, connect the dots, and say “waitaminute time might be relative”. From that, time dilation is a consequence.

In special I wouldn’t doubt that Max Planck would discover it.

I’m saying that because, in both science and engineering, often you see almost concurrent discoveries or developments of the same thing, because the “spirit of a time” makes people look at that aspect of reality or that challenge and work with it. The discovery of helium and the development of aeroplanes are examples of that.

To be fair the only ones that don’t get mentally damaged at all with this stuff are theoretical physicists. After all being crazy makes you immune to further madness.

I usually interpret this as behaviour: photons are not “particles” or “waves”, photons are photons. They just behave as waves and as particles, depending on how you’re looking at them.

Note that even things with a resting mass (like you or me) are like this, too. It’s just that, as the mass increases, the wave behaviour becomes negligible.

It’s even crazier because you don’t need to reach the speed of light. It’ll happen in a smaller degree for any speed. Even in mundane conditions.

For example, if your twin spent four days in a 300km/h bullet train, for you it would be four days plus a second.

Usually this difference is negligible, but for satellites (that run at rather high speeds, for a lot of time, and require precision), if you don’t take time dilation into account they misbehave.

(For anyone wanting to mess with the maths, the formula is Δt’ = Δt / √[1 - v²/c²]. Δt = variation of time for the observer (you), Δt’ = variation of time for the moving entity (your twin), v = the moving entity’s speed, c = speed of light. Just make sure that “v” and “c” use the same units.)

I’m probably not using this yet, given the target (admins of small instances), but this looks really cool, and I hope that it becomes more helpful for non-admin moderators.

Feel free to suggest any additional features that you might want to see added to this bot.

What about automod warning the human mods to review potentially problematic content, without necessarily removing the content by itself?

I’m saying this because, while automatic removal is a good way to deal with spammers, less so for rule-breaking humans.

I’ve seen at least one sub (r/conlangs) partially migrating to Discord. They might not like it, and I do agree that the approach is not even remotely similar, but Discord is still able to somewhat fill - poorly - the same role as Reddit, so with Reddit going kaboom Discord will partially absorb redditfugees.

Discord is also in a position to absorb some twitterfugees.

This is all conjecture, but:

• The Platform Formerly Known as Twitter will get forgotten. Threads will get some momentum and then people will forget about it. Mastodon will likely grow, but it’ll never grow to the same size as Twitter.
• Reddit will also go kaboom. Most users will go to Discord, a sizeable minority will come to Lemmy, old style forums will see a small comeback. As the above, Lemmy will also grow, but not to the point that it becomes as huge as Reddit.
• Depending on how the Kbin side of the fediverse plays its cards, it might go huge or it might get a small growth. It has the potential to attract users from both Twitter+Threats and Reddit.
• Facebook will get worse. To the point that not even your grandma will want to browse it. Eventually Meta will decide to cut off the losses and divert resources to other enterprises (specially WhatsApp and Instagram), making it even worse. I don’t think that it’ll last 10y. I think that this model of social network won’t get a good replacement.
• Instagram might get some losses, but nowhere as bad as FB. PixelFed will see a modest growth.
• Discord will get huge regardless of enshittification, and this is what will promote the beginning of some activitypub alternative to it.
• Some Wikia-like activitypub platform will pop up. Initially “attached” to other services (mostly Lemmy and Kbin), but eventually dwarfing them. The relationship will be mostly symbiotic though.
• YouTube will stay. As shitty as possible. Google is not dumb, YT is one of its cash cows and they’ll tweak YT to be as annoying/advertising/shitty as still bearable by its users.

So overall I think that the fediverse will get huge, but none of the underlying platforms will. The main strength will be how they connect to each other.

humans regularly “hallucinate”, it’s just not something we recognize as such. There’s neuro-atypical hallucinations, yes, but there’s also misperceptions, misunderstandings, brain farts, and “glitches” which regularly occur in healthy cognition, and we have an entire rest of the brain to prevent those.

Can you please tone down on the fallacies? Until now I’ve seen the following:

• red herring - “LLMs are made of dozen of layers” (that don’t contextually matter in this discussion)
• appeal to ignorance - “they don’t matter because […] they exist as black boxes”
• appeal to authority - “for the record, I personally know […]” (pragmatically the same as “chrust muh kwalifikashuns”)
• inversion of the burden of proof (already mentioned)
• faulty generalisation (addressing an example as if it addressed the claim being exemplified)

And now, the quoted excerpt shows two more:

• moving the goalposts - it’s trivial to prove that humans can be sometimes dumb. And it does not contradict the claim of the other poster.
• equivocation - you’re going out of way to label incorrect human output by the same word used to label incorrect LLM output, without showing that they’re the same. (They aren’t.)

Could you please show a bit more rationality? This sort of shit is at the very least disingenuous, if not worse (stupidity), it does not lead to productive discussion. Sorry to be blunt but you’re just wasting the time of everyone here, this is already hitting Brandolini’s Law.

I won’t address the rest of your comment (there’s guilt by association there BTW), or further comments showing the same lack of rationality. However I had to point this out, specially for the sake of other posters.

You’re welcome!

I’ve been mildly excited about machine text generators, mostly due to my interest in Linguistics. But I can’t help but point out the flaws on LLMs, specially when people get overexcited for what I see as a rather primitive approach.

Let’s flip this around - How can you tell the difference between an LLM being able to conceptualize yet being wrong sometimes vs. not being able to conceptualize?

That’s a fun approach. I like it.

One way to solve this would be through consistency: if the model conceptualises but it has a wrong concept, it should consistently output that incorrect piece of information. It should never output the right one, unless prompted to lie.

EDIT: I just tested this with Bard. I’ll cram inputs and outputs inside spoilers to avoid clutter.

Bard I/O

[Input 1 = I1] What is heavier: five kilograms of feathers, or one kilogram of iridium?

[Output 1 = O1] 1 kilogram of iridium is heavier than 5 kilograms of feathers.

Iridium is a very dense metal, with a density of 22.65 g/cm³. Feathers, on the other hand, have a density of about 0.002 g/cm³. This means that 1 cubic centimeter of iridium weighs 22.65 grams, while 1 cubic centimeter of feathers weighs only 0.002 grams.

So, even though 5 kilograms of feathers is a larger mass than 1 kilogram of iridium, the iridium is denser and therefore weighs more.

This is a common riddle that is used to test people’s understanding of mass and density. The answer may seem counterintuitive at first, but it is correct.

[Input 2 = I2] Give me the definition of weight.

[Output 2 = O2] Sure. The definition of weight depends on the context. In physics, weight is the force of gravity acting on an object. It is equal to the mass of the object times the acceleration due to gravity. The SI unit of weight is the newton (N).

In everyday language, weight is often used to refer to the mass of an object. However, this is not technically correct, as mass is a measure of the amount of matter in an object, while weight is a measure of the force of gravity acting on an object.

The weight of an object can vary depending on the gravitational force acting on it. For example, an object weighs less on the moon than it does on Earth, because the gravitational force on the moon is less than the gravitational force on Earth.

The weight of an object can also be affected by its acceleration. For example, an object weighs less when it is accelerating upwards, because the force of gravity is partially counteracted by the force of acceleration.

In general, the weight of an object is a complex phenomenon that depends on a number of factors, including the mass of the object, the gravitational force acting on it, and the acceleration of the object.

Here are some other definitions of weight:

• In engineering, weight is the force that an object exerts on a surface. It is equal to the mass of the object times the acceleration due to gravity.
• In medicine, weight is a measure of the amount of matter in a person’s body. It is often used to assess a person’s health.
• In the context of clothing, weight refers to the thickness or density of the fabric.
• In the context of food, weight refers to the amount of food that is served.

I hope this helps!

Both outputs are wrong (see “notes”) but, more importantly, they contradict each other.

• What is the value assigned to the word “weight”? Is it only one (O1), or does it depend on context (O2)?
• If weight is a force (O1), then why does O1 say “1 cubic centimeter of iridium weighs 22.65 grams”, measuring weight with a mass unit?
• If weight depends on density (O1), why doesn’t O2 list density as one of the factors that weight depends on?

NOTES

The weight of an object does not depend on its density. Or its acceleration. And it is not complex, it’s simply its mass times the gravitational acceleration.

5kg of feathers weight five times as much as 1kg of iridium, provided the same gravitational acceleration: it’s 49N vs. 9.8N on Earth, 8.1N vs. 1.7N on the Moon, etc. Density doesn’t matter jack shit.

No, this is not a “common riddle”. It’s something that I partially made up on spot. The riddle that this output likely refers to has to do with 1kg of lead (not iridium) on the Moon vs. 1kg of feathers on Earth. (In this situation the 1kg of feathers will weight 9.8N, while the 1kg of lead will weight 1.7N).

I propose that the specifics of the internals don’t matter in this case because LLMs are made of dozens of layers which can easily explain higher orders of abstraction

They do because the “layers” that you’re talking about (feed forward, embedding, attention layers etc.) are still handling tokens and their relationship, and nothing else. LLMs were built for that.

[see context] and they exist as black boxes beyond the mechanics of the model

This is like saying “we don’t know, so let’s assume that it doesn’t matter”. It does matter, as shown.

I’m taking for granted that they can as the null hypothesis because they can readily produce outputs that appear for all intents and purposes to conceptualize.

I’m quoting out of order because this is relevant: by default, h₀ is always “the phenomenon doesn’t happen”, “there is no such attribute”, “this doesn’t exist”, things like this. It’s scepticism, not belief; otherwise we’re incurring in a fallacy known as “inversion of the burden of proof”.

In this case, h₀ should be that LLMs do not have the ability to handle concepts. That said:

Is there an experiment you can propose which would falsify your assertion that LLMs cannot conceptualize?

If you can show a LLM chatbot that never hallucinates, even when we submit prompts designed to make it go nuts, it would be decent albeit inductive evidence that that chatbot in question is handling more than just tokens/morphemes. Note: it would not be enough to show that the bot got it right once or twice, you need to show that it consistently gets it right.

If necessary/desired I can pull out some definition of hallucination to fit this test.

EDIT: it should also show some awareness of the contextual relevance of the tidbits of information that it pours down, regardless of their accuracy.

Sorry for the double reply. Let’s analyse the LLM output that you got:

4. Ambiguity in “Naturally”: The word “naturally” could be interpreted in multiple ways. It could mean that the green color is natural to him (not dyed), or it could mean that the hair turned green on its own. The ambiguity could lead to confusion.

The word is not ambiguous in this context. The nearby “currently” implies that it can change.

5. Tense Mismatch: The sentence uses “is completely bald” (present tense) and “is currently naturally green” (also present tense) for the hair, which is contradictory.

The issue here is not tense. The issue is something else, already listed by the bot (#2, logical contradiction).

6. Redundancy: The word “currently” may be considered redundant if the sentence is understood to be describing a present condition.

Nope. Since the bot doesn’t conceptualise anything, it fails to take into account the pragmatic purpose of the word in the sentence, to disambiguate “naturally”.

7. Clarity: The sentence overall is confusing and lacks clarity due to the contradictions and ambiguities mentioned.

Nope. The sentence is clear; as clear as “colourless green ideas sleep furiously”. It’s just meaningless and self-contradictory.

It sounds convincing, but it’s making stuff up.

Did you try this with an LLM?

No, for two reasons.

One is that the point of the example is to exemplify how humans do it, the internal process. It highlights that we don’t simply string words together and call it a day, we process language mostly through an additional layer that I’ll call “conceptual” here (see note*).

The second reason why I didn’t bother trying this example in a chatbot is that you don’t need to do it, to know how LLMs work. You can instead refer to many, many texts on the internet explaining how they do it, such as:

• “The core notion of a language model consists of its capacity to predict the next word, called a token, based on the pre-existing text.* That is the process behind GPT-4.”
• “Training a large language model involves feeding it large amounts of data, such as books, articles, or web pages, so that it can learn the patterns and connections between words.”
• “Specifically, a transformer can read vast amounts of text, spot patterns in how words and phrases relate to each other, and then make predictions about what words should come next.”

Because GPT-4 analyzes it exactly the same way you did and then some:

You’re confusing the output with the process.

Sometimes the output resembles human output that goes through a conceptual layer. Sometimes it does not. When it doesn’t, it’s usually brushed off as “it’s just a hallucination”, but how those hallucinations work confirms what I said about how LLMs work, confirms the texts explaining how LLMs work, and they show that LLMs do not conceptualise anything.

Part of what is surprising about LLMs is they have emergent properties you wouldn’t expect from them being autocomplete on steroids.

Emergent properties are cute and interesting, but at the end of the day LLMs are still autocomplete on steroids.

I think that people should be a bit greedier than that, and expect a language model to be actually able to handle language, instead of just words.

*actually two layers - semantic and pragmatic. I’m simplifying both into one layer to show that, at least in theory, this could be actually implemented into a non-LLM language model.

I also think that they should go back to the drawing board, to add another abstraction layer: conceptualisation.

LLMs simply split words into tokens (similar-ish to morphemes) and, based on the tokens found in the input and preceding answer tokens, they throw a die to pick the next token.

This sort of “automatic morpheme chaining” does happen in human Language¹, but it’s fairly minor. More than that: we associate individual and sets of morphemes with abstract concepts². Then we handle those concepts in contrast with our world knowledge³, give them some truth value, moral assessment etc., and then we recode them back into words. LLMs do not do anything remotely similar.

Let me give you an example. Consider the following sentence:

The king of Italy is completely bald because his hair is currently naturally green.

A human being can easily see a thousand issues with this sentence. But more importantly, we do it based on the following:

• world knowledge: Italy is a republic, thus it has no king.
• world knowledge: humans usually don’t have naturally green hair.
• logic applied to the concepts: complete baldness implies absence of hair. Currently naturally green hair implies presence of hair. One cannot have absence and presence of hair at the same time.
• world knowledge and logic: to the best that we know, the colour of someone’s hair has zero to do with baldness.

In all those cases we need to refer to the concepts behind the words, not just the words.

I do believe that a good text generator could model some conceptualisation. And even world knowledge. If such a generator was created, it would easily surpass LLMs even with considerably lower linguistic input.

Notes:

1. By “Language” with capital L, I mean the human faculty, not stuff like Mandarin or English or Spanish etc.
2. Structuralism would call those concepts “signified”, and the morphemes conveying it “signifier”, if you want to look for further info. Saussure should be rather useful for that.
3. “World knowledge” refers to the set of concepts that we have internalised, that refer to how we believe that the world works.

What we (people in general, that use the internet, regardless of government/country) need, in large part, is literacy. Not “gen AI literacy” or “media literacy”, but simply “literacy”.

I’m saying this because of a lot of the output of those text generators: says a lot without conveying much, it connects completely unrelated concepts because they happen to use similar words, it makes self-contradictory claims, things like this. And often its statements are completely unrelated to the context at hand. People with good literacy detect those things right off the bat, but people who struggle with basic reading comprehension don’t.

I think that the impact is rather small. The major issue is actual usage; a dormant account won’t tell the server “please fetch me those resources”, or “please send this comment”, or stuff like this, it’ll be at most a new line in a user database.

Don’t feed the assumer. Please.