A new classroom resource should not be recommended for wider adoption simply because teachers like it or students appear engaged. Before making the case to school leaders, collect a manageable mix of evidence showing whether the tool addresses a defined instructional problem, supports effective teaching and improves student learning in your setting.
The strongest recommendation combines four perspectives:
Is the tool addressing a real and clearly defined problem?
Are teachers able to use it as intended?
Are students participating meaningfully?
Is there evidence of improved learning or progress?
This approach gives Maya a credible recommendation without turning a short pilot into a burdensome research project.
Start by Defining the Instructional Problem
Begin with the problem, not the product. A tool may be well designed but still be a poor fit if it does not address the school’s most important need.
Write a one-sentence problem statement using this structure:
Examples
– Year 9 students can identify key information in non-fiction texts but struggle to explain how evidence supports an argument.
– Students complete independent practice inconsistently, and teachers spend significant lesson time identifying misconceptions.
– Teachers lack a consistent way to provide timely feedback on extended responses.
– Students participate in discussion unevenly, with a small number of pupils contributing most of the time.
Avoid broad goals such as “improve engagement” or “raise attainment.” Define what better learning would look like in observable terms.
Identify the active ingredients
Clarify what the tool is expected to change. These elements are sometimes called the tool’s active ingredients: the features or practices that are supposed to produce the benefit.
For example, a resource may be expected to improve learning by:
– Providing immediate feedback after practice questions
– Structuring peer discussion with prompts
– Making misconceptions visible to the teacher
– Breaking complex tasks into sequenced steps
– Increasing opportunities for retrieval and deliberate practice
If the active ingredient is unclear, it will be difficult to interpret the evidence. A positive result may be attributed to the tool when the real cause was additional teacher explanation, extra lesson time or increased attention during the pilot.
Establish a Baseline Before the Pilot
A baseline shows what was happening before teachers introduced the resource. Without one, it is difficult to determine whether observed changes represent genuine improvement, normal variation or a change in teaching conditions.
Choose a small number of baseline measures that directly relate to the instructional problem.
Useful baseline evidence
– A common assessment or short diagnostic task
– Existing attainment or progress data
– Work samples scored against a shared rubric
– Attendance or completion data where relevant
– A brief student survey
– A short teacher workload or confidence rating
– Observation notes focused on the target behaviour
– The number of students participating, completing or responding during a typical lesson
Do not collect every possible measure. Two or three well-chosen baseline indicators are usually more useful than a large dataset that no one has time to interpret.
Make the comparison fair
Record relevant context alongside the baseline:
– Year group, subject and class size
– Number of lessons or weeks involved
– Teacher experience with the topic
– Other interventions introduced at the same time
– Assessment conditions
– Student attendance or timetable changes
– Whether the comparison group used the usual resource or practice
A simple pre- and post-pilot comparison can be useful for local decision-making, but it does not prove that the tool caused the change. State this limitation clearly in the recommendation.
Track Teacher Use Without Creating Extra Paperwork
Implementation evidence explains whether the tool was actually used in a way that could reasonably produce the intended result. If outcomes are weak, implementation data helps distinguish between an ineffective tool and a tool that was difficult to use or insufficiently supported.
Monitor only the features that matter to the decision.
Use a minimum viable implementation measure
A short weekly check-in could ask teachers to record:
– Which lessons used the tool
– How long it was used
– Which core feature was used
– Whether students completed the intended task
– One barrier or adaptation, if applicable
This can be captured through an existing planning document, shared form, learning platform or staff meeting check-in. Avoid asking teachers to write lengthy reflections after every lesson.
A useful format is:
Measure exposure, adherence and quality separately
These measures answer different questions:
How often and for how long did students use the tool?
Were the essential features used?
Did the tool support meaningful instruction, or was it used as a superficial add-on?
What did teachers change, and why?
A teacher may use a resource frequently but not use its most important feature. Conversely, a teacher may adapt it substantially while preserving the underlying instructional purpose.
Use a short observation or review sample rather than observing every lesson. Evidence on implementation monitoring recommends making measures observable, measurable and complete, and deciding in advance whether information will come from observation, materials or staff self-report.[1]
Gather Evidence of Student Engagement
Student engagement should not be reduced to whether pupils appear busy or enjoy using a resource. Collect evidence of **behavioural, cognitive and emotional engagement**.
Behavioural engagement
Look for observable participation:
– Students begin the task promptly
– Students complete the intended activity
– More students contribute to discussion
– Students persist after an incorrect answer
– Students use feedback to revise their work
– Fewer students are off task
Cognitive engagement
Look for evidence of thinking and learning:
– Students explain their reasoning
– Responses become more detailed or accurate
– Students make connections between concepts
– Students identify and correct misconceptions
– Students use subject-specific vocabulary
– Students apply a strategy in a new context
Student voice
Use a brief survey or exit question rather than a lengthy questionnaire. For example:
– The resource helped me understand what I was learning.
– The instructions helped me work independently.
– The feedback helped me improve my answer.
– I knew what to do when I made a mistake.
– What helped you most?
– What made learning harder?
Use a small number of response options and include at least one open question. Compare student perceptions with work samples and assessment evidence; positive reactions alone should not determine adoption.
Collect Evidence of Learning
Learning evidence should be aligned with the problem statement and collected using measures that teachers already understand.
Recommended evidence sources
– A common pre- and post-assessment
– Short low-stakes checks for understanding
– Work samples from before and during the pilot
– A shared rubric applied by more than one teacher
– Delayed tasks to test retention
– Transfer tasks requiring students to apply learning in a new situation
– Student explanations, worked examples or written reasoning
Where possible, include a comparison with usual practice. This may be another class, a previous unit or a parallel task completed without the new tool. Record the limitations of the comparison rather than presenting it as a definitive causal evaluation.
Look beyond average scores
Disaggregate findings where useful:
– By year group or class
– By prior attainment
– By attendance
– By student group
– By level of tool exposure
– By teacher or implementation model
Check whether the tool appears to help only students who are already confident. Also examine whether it supports students who need additional scaffolding, while avoiding claims that cannot be supported by the available sample.
Combine quantitative and qualitative evidence
A credible local evaluation might include:
– A change in assessment performance
– A sample of marked student work
– A short student survey
– Teacher use data
– A small number of observations
– A record of barriers and adaptations
This mixed-evidence approach helps explain not only whether results changed, but how and why.
Record Implementation Barriers
A tool cannot be fairly assessed if barriers are treated as individual teacher failure. Record the conditions that affected use.
Common barriers include:
– Limited planning or training time
– Unclear guidance
– Technical or login problems
– Inadequate device access
– Timetable constraints
– Difficulties adapting materials for additional needs
– Misalignment with the curriculum or assessment
– Insufficient subject-specific content
– Duplicate work in existing systems
– Lack of follow-up coaching
– Student unfamiliarity with the routine
For each barrier, record:
1. What happened?
2. How often did it occur?
3. Which classes or teachers were affected?
4. What was tried?
5. Was the issue resolved?
6. Would the issue remain at wider scale?
This turns anecdotal concerns into practical implementation evidence. It also helps school leaders distinguish between a problem that can be solved through preparation and one that makes wider adoption unrealistic.
Combine Local Findings With External Research
Local evidence tells you whether the tool appears feasible and useful in your context. External research helps you judge whether the underlying approach has broader support and whether the local findings are plausible.
Use external evidence to ask:
– Has the underlying instructional approach been studied?
– Were studies conducted in settings similar to yours?
– Were outcomes measured using credible assessments?
– Was implementation described clearly?
– Were findings replicated across more than one setting?
– Does the research address the same student age range and subject?
– Are claims based on independent evidence or vendor materials?
A research-supported approach is not automatically effective in every school. Context, implementation quality and student needs still matter. The Institute of Education Sciences notes that stronger evidence generally requires well-designed studies across multiple typical school settings, including settings similar to the schools making the decision.[2]
Treat external research as one part of the decision, not a substitute for local testing. Implementation guidance recommends examining whether a proposed approach is evidence-informed, suitable for the setting and feasible to implement, then monitoring and adapting it as it is used.[3]
Use a Practical Pilot Design
A manageable pilot should answer a defined decision question within a fixed period.
Define the pilot
Agree in advance on:
– Participating subjects, year groups or classes
– Pilot length
– Core tool features to test
– Expected teacher support
– Baseline measures
– Outcome measures
– Implementation measures
– Review date
– Decision criteria
A pilot should be long enough for teachers and students to become familiar with the routine, but not so long that the school continues investing in a weak approach without review.
Keep the teacher workload proportionate
To reduce friction:
– Use assessments already planned where possible
– Build checks into normal lesson routines
– Use one shared implementation form
– Ask for short responses rather than narrative logs
– Sample observations instead of observing every lesson
– Provide a model lesson or example activity
– Schedule one midpoint check-in
– Give teachers a clear explanation of how findings will be used
The goal is to evaluate the resource, not create a parallel reporting system.
Downloadable One-Page Evidence-Planning Worksheet
Copy the worksheet below into a document and save it as a one-page PDF or staff template.
—
# Instructional Tool Evidence-Planning Worksheet
**Tool or resource:** ___________________________________________
**Lead evaluator:** _____________________________________________
**Pilot dates:** ________________________________________________
**Classes, subjects or year groups:** _____________________________
## 1. Instructional problem
What specific learning or teaching problem are we trying to address?
__________________________________________________________________
What evidence shows that this problem exists?
__________________________________________________________________
## 2. Intended change
What should the tool change in teaching, student behaviour or learning?
__________________________________________________________________
What are the tool’s essential features or active ingredients?
__________________________________________________________________
Before recommending wider adoption, ask:
1. Did the tool address the original instructional problem?
2. Did students show stronger learning evidence than at baseline or under usual practice?
3. Were improvements visible across more than one class, teacher or student group?
4. Did teachers use the essential features consistently enough to test the approach?
5. Were positive results linked to the tool rather than unrelated changes?
6. Was the workload acceptable for teachers and students?
7. Can the school provide the training, time, technology and support required at scale?
8. Does external research support the underlying approach?
9. Are the costs and risks proportionate to the likely benefit?
10. What would need to be true for wider adoption to succeed?
Do not require perfect evidence before making a practical decision. Instead, match the decision to the strength of the evidence. A promising but uncertain tool may justify a second, better-supported pilot; it does not automatically justify network-wide adoption.
Planning a classroom-resource evaluation? Download the one-page evidence-planning worksheet, agree on a small set of measures with participating teachers and schedule the decision review before the pilot begins. This gives school leaders usable evidence while keeping the evaluation focused on better instruction—not extra paperwork.
