A school board can buy generative AI licences in a week. It can celebrate a lively staff training day, launch a chatbot pilot and watch students produce more polished assignments. But none of those accomplishments answers the question that matters most: are students learning more?
The difference is not semantic. A tool may make work easier without helping a learner acquire a lasting capability. It may also make feedback more timely, help a teacher notice a misconception or give a hesitant student another way into a difficult idea. Both possibilities are real. What distinguishes them is how the tool is used—and what the school chooses to measure.
For school principals, international school networks and education leaders, the right question is no longer simply whether AI belongs in classrooms. It is whether AI-supported practice leaves students better able to understand, remember, explain and act when the tool is unavailable.
The uncomfortable difference between better work and better learning
Imagine two students who submit equally strong explanations of climate change. One asks an AI assistant to write the explanation, edits a few sentences and hands it in. The other drafts an initial account, uses AI to identify weaknesses, checks independent sources, revises the argument and explains the corrections to a classmate.
The documents might look similar. The cognitive work is different.
This distinction runs through the OECD Digital Education Outlook 2026: completing a task successfully with generative AI does not necessarily mean the user learned what was needed to complete it independently. A school that looks only at output quality may confuse a more capable system with a more capable student.
That does not make well-produced work irrelevant. It means it is one indicator, not the outcome itself. Learning also appears in what students can recover after a delay, adapt to a new problem, justify under questioning and improve after feedback.
When leaders ask whether an AI pilot is effective, the first response should be another question: Effective at what, compared with what, and for whom?
What the evidence shows in 2026: promise, warnings and context
The strongest response to AI hype is not pessimism. It is a willingness to examine different findings without forcing them into one universal conclusion.
When AI makes practice easier but independent performance falls
In a 2025 field experiment published in PNAS, researchers studied nearly 1,000 high-school mathematics students using different GPT-4-based tutors. Access to a general-purpose-style assistant improved performance on supported practice problems. But when students later took an exam without AI, the group using the more open-ended assistant performed worse than the control group. A more carefully scaffolded tutor, designed to offer hints rather than reveal solutions, substantially mitigated that negative result.
This is evidence about particular mathematics tasks, tools and conditions—not proof that all AI causes dependency. The key insight is more useful than an absolute judgment: design choices determine whether support strengthens thinking or replaces it.
When tutoring is promising but actual student engagement is thin
An August 2026 NBER working paper reported a two-year randomized study across 18 Tennessee middle schools using Khan Academy and the AI tutor Khanmigo during mathematics support sessions. Researchers found modest gains in mathematics achievement. They also observed that students often used the AI tutor only intermittently, and the gains resembled those associated with practice on the platform without AI.
That challenges an easy assumption in procurement meetings: making AI available is not equivalent to making it educationally consequential. Students may have access but not use it deeply. Teachers may have dashboards but little time to interpret them. A promising feature can coexist with ordinary learning habits.
When integration changes the result
Other findings are more encouraging. A 2025 randomized college study found that a purpose-built AI tutor supported higher learning outcomes in less time than the comparison active-learning class in that setting. But university students, a custom-built tool and a particular instructional design cannot stand in for every primary or secondary classroom.
Conversely, an October 2026 university trial involving 2,379 undergraduates reported lower grades and participation when an AI tutor was made available alongside an existing course. That study is a discussion paper, and its findings require careful interpretation. Yet it underlines a practical risk: adding help to a learning environment may displace beneficial activity rather than improve it.
A European Commission evidence review published in September 2026 likewise concludes that intelligent tutoring systems can support learning, but benefits vary with their instructional design and classroom integration.
The evidence does not support “AI always improves learning” or “AI always harms learning.” It supports a more demanding conclusion: leaders need to evaluate specific uses, specific outcomes and specific contexts.
Five questions every school AI pilot should answer
A credible AI strategy starts with a learning question and ends with observable evidence. The following scorecard is a practical decision aid, not a validated psychometric instrument.
| Question | What to examine | What would not be enough |
|---|---|---|
| 1. Did understanding improve? | Clearer concept explanations, better diagnosis of misconceptions and success on comparable tasks | More fluent AI-assisted assignments |
| 2. Can students work unaided? | Short independent checks aligned with the learning goal | High scores while AI remains available |
| 3. Does learning last? | Appropriate delayed recall or retention checks | Immediate enthusiasm after a lesson |
| 4. Does knowledge transfer? | Applying a principle to a new or unfamiliar problem | Repeating the AI tutor's original example |
| 5. Who benefits—and who is left behind? | Results by starting proficiency, support needs, accessibility and access conditions, using appropriate privacy safeguards | A school-wide average that conceals differences |
The fifth question matters particularly. A pilot can be positive on average and still disadvantage students with limited access, weaker prior knowledge or different support needs. It can also benefit students who previously struggled to receive feedback. Leaders need both kinds of evidence before scaling a program.
Teacher workload, inclusion, safety and student experience deserve separate monitoring. They are important outcomes, but they should not be reported as direct proof that academic learning improved.
Start with the lesson—not the licence
Suppose a school wants Year 8 students to become stronger at explaining mathematical reasoning. That goal suggests a sequence quite different from “use the chatbot whenever you get stuck.”
Before AI: each student makes an initial attempt, identifies what is known and explains one uncertain step.
With AI: the tool gives hints, asks a diagnostic question or presents a misconception for the student to inspect. It does not automatically provide the completed reasoning.
After AI: students attempt a different, comparable problem without the assistant and explain the underlying method.
Later: the teacher revisits the same concept after an appropriate interval to see whether the understanding was retained.
This sequence is not the only effective approach. Its value is that it makes the learning pathway observable. If the student still cannot select the method independently, the school knows the next need is instruction—not another impressive AI demonstration.
There are moments when allowing AI to do more of the work is entirely appropriate. Professional translation, accessible text adaptation, brainstorming alternatives and the study of AI-generated evidence may all have legitimate learning purposes. The boundary should follow the objective. If the objective is learning to reason, the reasoning cannot quietly be delegated away.
The leadership problem is also an organizational problem
Principals often inherit the consequences of AI rollouts rather than design them. A provider showcases a compelling platform. Families ask whether the school is falling behind. Teachers differ in confidence and philosophy. Data protection officers worry about student information. Students experiment with tools the school has not approved.
The temptation is to resolve these tensions with a single announcement: We are now an AI-ready school.
But readiness is not a slogan. It is a set of operational capacities.
A school should know who approves new tools, who checks age suitability and privacy, what teachers may upload, how permitted student use is communicated, how accessible alternatives are provided, and who reviews evidence when a pilot does not work. The UNESCO guidance on generative AI in education emphasizes human agency, age appropriateness, privacy and pedagogical validation—precisely the responsibilities that cannot be delegated to a vendor.
Teachers also need time to make instructional choices. A demonstration can show what a platform does. Professional learning must help teachers decide when a feature advances a learning goal, when it undermines one, and how to tell the difference.
For a closer look at this organizational challenge, see Learning Coach's guide to turning school AI policy into classroom routines. For the assessment question inside individual lessons, read When AI Gives the Right Answer: How Can Teachers Know Students Really Understand?.
A 90-day pilot built around evidence
A small, well-observed pilot is often more informative than a whole-school rollout with no baseline. Here is a possible 90-day sequence for a school leadership team.
Days 1–15: name the learning outcome
Choose one subject, age group and measurable capability, such as explaining a scientific mechanism, editing an argument with evidence or solving unfamiliar fraction problems. Record a baseline using comparable work that does not depend on the AI tool. Identify teacher time, access and safeguarding requirements. Avoid collecting unnecessary student data.
Days 16–45: test a defined teaching routine
Train participating teachers on a short, explicit protocol: what students do independently, what the AI may help with, and what they must demonstrate after assistance. Collect implementation notes so differences in how the tool is used do not disappear into an overall score.
Days 46–70: assess both support and independence
Look at AI-assisted work and independent performance. Include suitably delayed and transfer tasks where feasible. Compare outcomes with the baseline and, ideally, a carefully chosen comparison group. A simple before-and-after rise cannot by itself establish that AI caused the change: teaching time, student mix and task difficulty may also shift.
Days 71–90: decide what to expand, change or stop
Review academic outcomes together with teacher feedback, student perspectives, accessibility, privacy and cost. Identify whether the instructional approach worked for particular students or tasks. Expand only when the evidence justifies expansion. Revise or discontinue pilots that produce attractive outputs without reliable learning.
A school does not need to become a research university. It does need enough evaluation discipline to avoid mistaking adoption for impact.
The Learning Coach perspective: the learner remains the outcome
Learning Coach begins from a simple educational conviction: the ultimate measure of a learning system is not how much content it delivers or how efficiently it generates answers, but what a person becomes capable of doing.
This perspective is an editorial and pedagogical position, not a claim that every Learning Coach method has been independently validated. It encourages school leaders to protect four things as AI becomes more powerful:
Cognitive effort: students still need opportunities to make meaningful attempts and revise errors.
Metacognition: learners should notice what they know, what they do not know and when assistance helps.
Agency: children and adolescents should gradually become less dependent on constant external prompting.
Human relationships: teacher judgment, encouragement, discussion and trust remain central to the conditions in which learning happens.
AI can be a useful part of that environment. It cannot, by itself, define what a school should value.
What should the next school leadership event actually accomplish?
If a school network is bringing principals, teachers, parents or policymakers together to discuss AI, the program should not end with excitement about tools. A worthwhile keynote can create shared questions; a workshop can then turn those questions into assessment routines, governance decisions and small pilots.
Before choosing an education keynote speaker, ask whether participants will leave able to distinguish task performance from learning, identify safeguards, challenge weak evidence and frame a realistic next experiment. Those outcomes offer a better event brief than a request for “the newest AI trends.”
The question worth taking into the auditorium is not “How much AI does our school use?” It is “What can our students now do that they could not do before—and how do we know?”
For schools and education organizations planning a conversation that connects AI, learning science and student autonomy, explore Learning Coach's international keynotes and request a tailored proposal.
References
OECD (2026), Digital Education Outlook 2026. Evidence synthesis on generative AI, educational design and learning; general conclusions do not replace local evaluation.
Bastani et al. (2025), Generative AI Without Guardrails Can Harm Learning, PNAS. Field experiment with high-school mathematics students; task- and tool-specific findings.
Oreopoulos & Low (2026), One Click Away: AI Tutoring with Khanmigo in a Two-Year School Experiment, NBER working paper. Cluster-randomized evidence from Tennessee middle schools; working-paper findings.
AI Tutoring Outperforms In-Class Active Learning (2025), Scientific Reports. Randomized college study using a custom tutor; not direct evidence for all school grades.
Liu et al. (2026), The Effects of Course-Integrated AI Tutoring on Student Performance and Engagement, IZA discussion paper. Randomized university trial; preliminary paper.
European Commission (2026), The Impact of Intelligent Tutoring Systems in General Education. Systematic review of causal evidence in school education.
UNESCO (2023), Guidance for Generative AI in Education and Research. Guidance on governance, age appropriateness, agency and privacy.
