If you want your students to think more critically about AI, show them how it can get things wrong.
One of the simplest ways I've found to start this conversation with students is with a question that most Australian students should be able to answer instantly:
What is the capital city of Australia?
The answer is Canberra.
But this isn't really a lesson about Australian geography.
It is an opportunity to show students how AI works, why it can get things wrong and why they should never automatically trust an AI-generated answer.
A simple way to teach critical thinking and AI
Ask your students to put the question into an AI chatbot:
"What is the capital city of Australia?"
Most of the time, the AI will confidently give you the correct answer: Canberra.
But there is also usually a student who very confidently shouts:
"Sydney!"
Everyone laughs.
And that laugh creates an awesome opportunity to stop and unpack what just happened.
Instead of simply telling students that AI sometimes makes mistakes, ask them to think about why it might make a mistake.
How would an AI system work out the answer in the first place?
That is where the conversation gets interesting.
What is an AI hallucination?
An AI hallucination is when an AI system generates information that is incorrect, misleading or unsupported, while presenting it as though it were a legitimate answer.
This can be particularly difficult for students to recognise because AI doesn't necessarily tell you when it is guessing.
The response can be written clearly.
It can sound confident.
It can even provide an explanation that appears to make sense.
That does not mean it is correct.
For students who are increasingly using AI to find information, complete schoolwork or answer questions, this is an important lesson.
Show students how AI can get an answer wrong
When I explain this to students, I like to take the answer apart rather than simply saying, "AI hallucinates."
Ask them:
How would AI figure out the capital city of Australia?
At a basic level, generative AI systems learn patterns from enormous amounts of training data. They don't approach every question in the same way a student would.
They generate an answer based on patterns and probabilities learned from the information they have been trained on.
So imagine that the information available to an AI system included websites or other content incorrectly saying that Sydney was the capital city.
It might encounter conflicting information.
Some sources might say Canberra.
Others might incorrectly say Sydney.
The AI still needs to generate an answer.
And it may produce an answer that sounds completely confident even when the information behind it is inaccurate or conflicting.
That gives students a much better understanding of why AI outputs need to be questioned and checked.
Take away the "rose-coloured glasses"
One of the biggest problems I see with AI education is that students can sometimes treat AI as though it is much smarter than it actually is.
If an AI chatbot gives them a polished answer in seconds, it can be very tempting to assume it must know what it is talking about.
I want students to take off those rose-coloured glasses.
AI can be incredibly useful.
But useful does not mean infallible.
A confident answer is not automatically a correct answer.
And a fluent answer is not evidence that the AI actually understands what it is saying in the same way a person does.
Once students understand that, we can start having much more useful conversations about when they should use AI and what they need to do after they receive an answer.
What should students do with an AI-generated answer?
The goal isn't to teach students to reject everything AI produces.
That would be just as unhelpful as telling them to trust everything.
Instead, students need a process for evaluating AI outputs.
That might mean asking:
Does this answer make sense?
Students should pause rather than automatically copying an answer simply because it sounds convincing.
Can I verify it?
Students need to know how to check important information against reliable sources.
What evidence supports this answer?
An AI-generated response is not evidence by itself.
Students should learn to distinguish between information and evidence.
Could the AI be missing something?
Students need to consider whether there is context, nuance or information that the AI has not accounted for.
Is this something I should be thinking through myself?
This is particularly important in education.
If students use AI for every question, they can miss the opportunity to develop the underlying thinking skills that school is supposed to help them build.
AI literacy is more than learning how to prompt
This is why I think AI literacy needs to be much broader than teaching students how to write better prompts.
Prompting is a useful skill.
But students also need to understand the limitations of the technology they are using.
They need to know that AI can:
produce incorrect information
reflect bias in its training data
confidently present an inaccurate answer
leave out important context
generate information that needs to be independently checked
Most importantly, students need to develop the habit of questioning the output.
That is where AI literacy and critical thinking come together.
Why this matters beyond the classroom
The "capital city of Australia" example is deliberately simple.
There isn't much at stake if an AI gets this particular question wrong.
But the same problem becomes much more important when students are using AI to research an assignment, understand a health topic, make decisions, evaluate something they have seen online or decide whether a piece of information is true.
The skill we are teaching isn't:
"AI sometimes gets things wrong."
It is:
"When AI gives me information, I need to think about how I know it is true."
That is a much more valuable lesson.
Students are going to encounter AI-generated information well beyond the classroom. They are already seeing it in search results, social media, videos and other digital platforms.
They need the critical thinking skills to navigate that environment.
A practical AI literacy activity for schools
You don't need a complicated lesson plan or an expensive AI tool to start this conversation.
A simple question can be enough.
Ask students something they know the answer to.
Let the AI answer.
Then unpack the process.
What did it get right?
What could it get wrong?
Why did it give that answer?
How could we check it?
What would happen if we didn't know the answer ourselves?
That last question is particularly important.
If a student already knows the answer is Canberra, they can spot the mistake.
But what happens when AI gives them an answer about something they know nothing about?
That is where critical thinking becomes essential.
Students can't fact-check everything by relying on their own knowledge.
They need to know how to question information, identify reliable sources and recognise when an answer needs further investigation.
Students need to know when to question AI
AI literacy isn't just about knowing how to use these tools.
It is about knowing when to question them.
A student who understands that AI can hallucinate is less likely to blindly copy an answer.
A student who understands that AI can sound confident while being wrong is more likely to stop and check.
And a student who understands that AI is a tool rather than an unquestionable authority is better equipped to use it responsibly.
That is the kind of practical AI education I focus on when working with schools and students.
If your school is looking at how to build critical thinking and AI literacy into student learning, my AI literacy workshops for schools can help students understand how AI works, where it can go wrong and how to think more critically about the information it produces. Programs such as Truth or Algorithm and Think Before You Prompt explore these skills in practical, age-appropriate ways.
