Ask HN: GPT-4 usually hallucinates when a prompt starts with "Do you know", why?
If a prompt starts with "Do you know ..." GPT-4 exhibits a strong bias to answer with "Yes, ...", for example if I ask "Do you know the _ meme" it always respond with yes:
> Do you know the "puppy flying with a kettle" meme?
> Yes, the "puppy flying with a kettle" meme features a cute puppy that appears [...]
> Do you know the "fish swimming in the universal void with a god cat" meme?
> Yes, the "fish swimming in the universal void with a god cat" meme features a surreal and humorous image of a fish swimming in a vast, empty space [...]
Is this common with all LLMs?
I do have an hypothesis: If GPT-4 was trained with a substantial amount of internet conversations it could be because it's very uncommon for people to respond to questions on forums with "No sorry I don't know", instead the question/thread would just die without answers.