Hallucinations, knowledge cutoffs, and other real limitations every user should know.
AI tools are genuinely impressive and genuinely limited. Understanding the limits isn't pessimism — it's how you use the tools without getting burned.
Hallucination is the term for when AI generates information that sounds accurate but is false. This includes invented facts, fake citations, made-up statistics, and plausible but wrong details.
Why does this happen? AI language models don't look up facts — they generate text based on patterns in their training data. When asked a question, the model produces what a correct answer would statistically look like. Sometimes that's accurate. Sometimes it's a very convincing fabrication.
The problem isn't that AI gets things wrong — everything gets things wrong. The problem is that AI gets things wrong with complete confidence and no signal that it's uncertain. It won't say "I'm not sure" when it's fabricating; it will just fabricate fluently.
How to spot potential hallucinations: - Very specific claims (exact statistics, paper titles, quotes) are higher risk - Things that would require real-time knowledge are often wrong - Anything obscure or highly specific is less reliable than common knowledge - If it sounds too perfectly tailored to your question, double-check it
The fix: treat specific claims as leads to verify, not facts to trust.
Every AI model is trained on data up to a certain date — its knowledge cutoff. After that date, the model knows nothing about what happened in the world.
Ask about a news event from last month, a recently published paper, or a company that was founded after the cutoff, and the model will either say it doesn't know, guess based on what it knew before, or hallucinate something plausible.
Cutoffs vary by model. GPT-4o, Claude, and Gemini all have different cutoffs, and newer versions typically push the cutoff further forward. Some models (Perplexity, Grok, Gemini with search enabled) can access the web to get around this — but even then, they're searching rather than knowing.
Check the documentation for whichever model you're using if recency matters for your task.
Most AI tools start fresh with every new conversation. They don't remember that you had a conversation last week, don't know your preferences, and don't accumulate context over time.
This means you often need to re-establish context at the start of each session: who you are, what you're working on, relevant background. It can feel repetitive.
Some tools are adding memory features — ChatGPT has a memory option that can store facts across conversations, and Claude has a similar feature in some configurations. These are improving but still limited compared to human memory. Read the privacy implications before enabling them if that matters to you.
A standard AI chatbot reads and writes text. It cannot send emails, make purchases, browse the web, modify files, or interact with the world unless it's been specifically built to do so through tools and integrations.
This is changing fast. AI agents — versions of these models equipped with tools and the ability to take actions — are becoming more common. ChatGPT's operator and tool-use features, Claude's computer use capability, and various agent frameworks are expanding what AI can act on.
But the base models you're chatting with are text generators, not actors. Don't assume an AI has done something in the world just because it told you it did.
Ask the same question twice and you may get meaningfully different answers. AI models are probabilistic — they don't have a single fixed answer to any question. This is by design (it allows for creative, varied output) but it means you can't always rely on consistency.
For important decisions, asking the same question multiple ways and comparing answers is a reasonable sanity check.
One of the strangest features of current AI is that it has no reliable self-awareness about its own errors. It will not reliably flag when it's uncertain, when it's guessing, or when it's fabricated something. It sounds the same whether it's citing a real study or inventing one.
This is why verification matters. The model's confidence is not a signal of accuracy.
Have a follow-up question about this topic?
Ask AI