Key takeaways
- Completion rates and quiz scores show activity, not impact. Measure what people do differently at work and what changes for the business.
- Kirkpatrick's four levels (reaction, learning, behavior, results) give a simple structure; the Phillips method adds ROI as a fifth level.
- Choose the business metric and capture a baseline before you design the course, not after it launches.
- Combine business data, manager observation and detailed learning data (for example xAPI) to see the full picture.
- ROI (%) = (net program benefits divided by program costs) x 100, after isolating the effect of training from other factors.
Why completion rates are not enough
Most learning management systems make it easy to report how many people finished a course and what they scored on the final quiz. Those numbers are useful for compliance, but they say little about whether anyone works differently afterwards.
The gap matters more now that AI tools make it fast and cheap to produce courses. When content is cheap, it is tempting to optimize for volume and completions. The organizations that get real value from eLearning judge it by capability: can people do the job better, and does the business see the difference?
The Kirkpatrick model: four levels of evaluation
The Kirkpatrick model is the most widely used framework for evaluating training. Each level answers a different question:
| Level | Question | Example measures |
|---|---|---|
| 1. Reaction | Did learners find it relevant and engaging? | Short surveys, net promoter style ratings, comments |
| 2. Learning | Did they gain the knowledge and skills? | Pre- and post-tests, scenario scores, simulation performance |
| 3. Behavior | Do they apply it on the job? | Manager observation checklists, quality audits, system usage data |
| 4. Results | Did business outcomes improve? | Sales, error rates, handling time, safety incidents, customer satisfaction |
Most organizations measure levels 1 and 2 well and rarely reach 3 and 4. Planning those higher levels from the start is what turns training data into business evidence.
Calculating ROI with the Phillips method
Jack Phillips extended the Kirkpatrick model with a fifth level: return on investment. The core formula is simple:
Illustrative example: if a sales onboarding program costs 20,000 (development, licenses and learner time) and produces 50,000 in additional margin that you can attribute to the training, net benefits are 30,000 and ROI is 150%.
The hard part is attribution. Use one or more of these ways to isolate the effect of training:
- Control groups: compare a trained group with a similar group that has not been trained yet.
- Trend lines: compare performance after training with the trend that existed before it.
- Expert estimates: ask managers and participants what share of the improvement came from training, and adjust for their confidence.
Count all costs honestly, including learner time away from work, and convert benefits to money only when the link is credible.
Choosing the right metrics for each type of training
| Training type | Behavior to observe | Business results to track |
|---|---|---|
| Sales | Discovery questions asked, use of the sales process | Win rate, deal size, time to first sale |
| Onboarding | Tasks completed without help | Time to productivity, early attrition |
| Customer service | Call quality scores, correct procedures | Customer satisfaction, first-contact resolution, handling time |
| Compliance and safety | Safe behaviors observed, policy checks passed | Incidents, audit findings, fines avoided |
| Software rollout | Correct use of key features | Help-desk tickets, data errors, adoption rate |
Pick one or two business metrics per program. Too many metrics make it hard to tell what changed.
A 6-step measurement plan
- Start with the business goal. Agree with stakeholders which outcome the training should move.
- Define the behaviors. List what people must do differently to move that outcome.
- Capture a baseline. Record current performance before launch.
- Design for the behaviors. Use practice, scenarios and simulations that mirror real work.
- Collect data at every level. Reaction, learning, behavior and results, at agreed time points.
- Report and improve. Share results in business terms and update the course where it falls short.
Collecting the data: LMS, xAPI and business systems
No single system holds all the evidence. Combine several sources:
- LMS reports for enrolments, completions and scores. SCORM courses report a fixed, fairly basic set of data.
- xAPI data in a learning record store for detail such as choices made in scenarios, attempts in simulations or activity outside the LMS. See our guide to SCORM vs xAPI vs cmi5.
- Business systems such as CRM, help desk, quality and HR systems for behavior and results.
- Surveys and manager checklists 30 to 90 days after training to capture on-the-job application.
Measuring AI-assisted training
Generative AI can speed up scripting, quiz writing and media production, but faster production does not guarantee better learning. To keep quality high:
- Assess with realistic scenarios and simulations rather than recall questions alone.
- Have subject-matter experts review AI-drafted content for accuracy and context.
- Compare behavior and results for AI-assisted courses with your existing courses, not just completion rates.
Common mistakes to avoid
- Deciding what to measure after the course has launched.
- Reporting only completions and satisfaction to leadership.
- Claiming all improvement as a training result without isolating other factors.
- Measuring too early, before people have had time to apply new skills.
- Collecting data nobody uses to improve the course.
Tell us the business result you need, and we will design training and measurement around it.
Get a free estimateRelated services
Frequently asked questions
What is the best way to measure eLearning effectiveness?
Use a framework such as the Kirkpatrick model to measure reaction, learning, behavior on the job and business results, and agree the business metric and baseline before the course launches.
How do you calculate the ROI of training?
Subtract program costs from the monetary benefits attributed to training, divide by program costs and multiply by 100. Isolate the training effect with control groups, trend analysis or expert estimates.
Are completion rates a good measure of training success?
Completion rates show participation, not impact. They are useful for compliance but should be combined with measures of skill, behavior and business results.
How long after training should you measure impact?
Measure learning immediately, behavior after 30 to 90 days, and business results over a period long enough for the metric to change, often one to two quarters.
Can xAPI help measure training impact?
Yes. xAPI records detailed learning activity, such as decisions in scenarios and activity outside the LMS, which can be combined with business data for a fuller picture.
Want training you can prove works?
Tell us the business result you need, and we will design training and measurement around it.
Get a free estimate- [email protected]
- +91 70161 48402
- Ahmedabad, India · serving clients worldwide