AI's hallucinations, Tolkien edition

[Ricardo]

I was bored, so I started having a Tolkien talk with ChatGPT and Gemini.

Boy were they hallucinating?

With ChatGPT I was discussing the fate of Tuor and Idril. ChatGPT told me that Christopher Tolkien explicitly cited a note/letter whatever from his father saying that Tuor and Idril reached Aman.

Then I started a separate chat and asked it to tell me where CT said that, and it answered that CT never said that, AI liar.

Then I asked Gemini a very simple question, sorry about my obsession but it basically described Idril saying she had silver hair and was called Celebrindal because of that.

I corrected it stating she had blond hair and Celebrindal was because she used to always be barefoot.

It answered, you are right in both accounts, that she inherited her blond hair from her mother blah blah blah.

If you want to learn something, forget it. The problem here is clearly that there isn't big enough dataset about Tolkien, fair enough but they answered with such certainly that unless you know the topic very well, you will think what the said is right.

In short, if you know the topic, you don't need them, if you don't, you can't trust them.

I wonder what other blunders they make. Zero out of 2.

 

[d4rk3lf]

I am not that annoyed that AI makes mistakes, but I am very annoyed how it present it as a facts, and even trying to defend it, until you really push it, and only then it admits that it made a mistake.
That's horrible in my opinion, and could be very dangerous.

It's not only Tolkien.. it's about everything... I asked about software that I personally know inside out, and it gave very wrong answers, and even trying to defend statements, until pushed.

 

[Deimos]

As has been stated elsewhere in the forum, AI "locks onto" keywords and starts searching for them, pulling in anything and everything ever mentioning these words in any medium.
Bear in mind that in that huge bucket of statements, questions and phrases are legit facts, urban myths, conjectures, opinions, hypotheses, pro-and-con arguments, and flat out errors.

Then AI [laughably] tries to make sense of all that.
So it ends up stringing together a lot scholarly sounding statements about JRRT, his works, CRRT, ME, HoME etc.
Read enough of it (and you don't need to read much), you will find AI will here and there flat out contradict its own previous assertion on some topic.

Which is why it's waste of time if someone seriously engages AI not just on Tolkien but on any topic whatsoever.
You can't tell if what it's telling you is totally true, partially true or totally false. It's a monstrous example of GIGO.
There is absolutely no way to tell (if you are starting from total or partial ignorance of a subject) what is garbage and what isn't.

(And just to keep you awake at night, know that insurance companies are pushing to use AI in place of human doctors for diagnosing illnesses.
Pleasant dreams 😱)

 

[Ricardo]

Well, the thing is that behind the cool name AI, in reality all you have is a very fast data mining. What the call LLM is really large datasets. Even to logically link information they use more information.

AI is only reliable when it relies on very specific datasets for very specific purposes.

The true is, the human brain does work like that in many situations, you find a solution because you have seen it before, and professional judgement is about how to link that prior experience and information together, and realizing there is a match with something you've seen before.

What AI doesn't have (and never will have) is intuition, imagination, creativity, all part of human intelligence.

 

[Deimos]

@Ricardo ...but again... I have posed a question to AI in impeccably correct grammar, gotten an answer, reworded the question to ask exactly the same thing (again in impeccably correct grammar) and received an answer that unambiguously contradicts the first answer.
How do you explain that?
And the fact that it does happen, you can see why I regard AI as 100% unreliable when used to the exclusion of other sources of info.
If you have other, reliable sources of info, why would you even waste a nanosecond with an "information source" the veracity of which and when, you have absolutely no way of knowing?
It may be able to rapidly mine data until the cows come home, but if it randomly spews out potentially unreliable answers, what good is it?

 

[Ricardo]

@Ricardo ...but again... I have posed a question to AI in impeccably correct grammar, gotten an answer, reworded the question to ask exactly the same thing (again in impeccably correct grammar) and received an answer that unambiguously contradicts the first answer.
How do you explain that?
And the fact that it does happen, you can see why I regard AI as 100% unreliable when used to the exclusion of other sources of info.
If you have other, reliable sources of info, why would you even waste a nanosecond with an "information source" the veracity of which and when, you have absolutely no way of knowing?
It may be able to rapidly mine data until the cows come home, but if it randomly spews out potentially unreliable answers, what good is it?

Yes, the problem with AI is that it designed to look as intelligence, but it's not.

The reasons why you can see different answers to different questions is that AI only understands content based on comparison with the models. While different words with the same meaning might be understood as the same for a human brain, it's not so for AI, one format matches some responses, the other matches others. It's similar to a child that doesn't know a lot of words, he/she might understand one way but not the other.

The reasons why people are using AI instead of other sources are many: it's cool, it's fast and sadly, it's not the only source that is wrong, many "reputable" sources are now not reliable simply because they just rely on other sources. I know a concrete case of the BBC disseminating fake information because it was fake at a "reliable" source.

On top of that, AI can be fed with wrong information with the purpose of misleading users. Most AI agents use interactions with users to add to their LLMs. Using bots to talk to AI may it possible to make them biased.

 

[Squint-eyed Southerner]

What AI doesn't have (and never will have) is intuition, imagination, creativity, all part of human intelligence.

By coincidence, I saw a news story this morning I was about to post in the Watching thread, but this seems like a better place.

Wherever AI is at the moment, we're doomed. 😳

DOOMED!

 

[Deimos]

Only music? How 'bout this?

 

[Ricardo]

By coincidence, I saw a news story this morning I was about to post in the Watching thread, but this seems like a better place.

Wherever AI is at the moment, we're doomed. 😳

DOOMED!

I can't see that video, not available in my country (Canada).

But I am seeing very worrying things happening. Grog was spreading nazi propaganda, Claude was threatening employees.

Wall Street is obsessed with AI and is pushing companies to it, making them make the wrong decisions, that's what I admire of Apple, they only use it where it makes sense and they are not pushed around by investors.

From this point of view, AI is the same as cryptocurrency, basically a hype fueled technology that is not needed (at least not the way is being used).