Your Course Completion Rate Does Not Always Measure The Right Things
Sooner or later, every conversation about online courses gets to completion rates. The number sounds like a verdict. Low completion means a bad course, a bad teacher, or proof that people don’t learn online.
I have been building a learning management system for online schools since 2019, and I have come to distrust that verdict. Not because completion doesn’t matter. It matters a lot. But a completion rate is a fraction, and almost all the arguing happens about the top half of it. The bottom half, the people you count as „everyone who started,“ quietly decides most of the result.
Here is what the research says about that bottom half.
Same Courses, Three Different Completion Rates
In 2024, Berkan Celik and Kursat Cagiltay published a study in Open Praxis that I wish every course creator would read [1]. They took four MOOCs on a Turkish platform, 15,805 enrollments in total, and calculated completion three ways.
- Counted against everyone who enrolled, completion was 30.02%.
- Counted only against active learners, the people who actually started, it was 43.08%.
- Counted against learners who said at the start that they intended to finish, it was 48.13%.
Same courses with the same instructors on the same platform. An 18-point swing, produced entirely by the choice of denominator.
If you have ever compared your course with a number from someone’s blog post, ask yourself which of these three they were quoting. Usually they don’t say.
Who Is In The Denominator Matters More Than The Content
The bigger version of this story comes from Justin Reich and José Ruipérez-Valiente, who analyzed 5.63 million learners in MIT and Harvard MOOCs and published the results in Science in 2019 [2].
In 2017-18, 3.13% of all participants completed their courses. Among „verified“ learners, the ones who paid, completion was 46%.
That is not a small gap. It’s roughly fifteen times. And both groups took the same courses. The content didn’t change between them. What changed was who they were and why they came. Someone who paid, planned, and cleared an evening for the course is a different learner from someone who clicked „enroll“ on a free course during lunch.
For anyone running a paid course, this cuts both ways.
The good news: the scary single-digit completion numbers you see in headlines mostly describe free, open enrollment. They are the wrong benchmark for a paid program.
The less comfortable news: a free lead-magnet course with low completion tells you almost nothing about your paid course. And a paid course with low completion can’t hide behind those headlines either.
Why The Standard Fixes Disappoint
When completion looks bad, the usual reflex is to add nudges. Reminder emails. Goal-setting prompts at the start. An accountability buddy.
Rene Kizilcec, Justin Reich, Michael Yeomans and colleagues tested exactly this kind of intervention on about 250,000 students in 250 courses from Harvard, MIT and Stanford [3]. Plan-making and related prompts did raise engagement in the first weeks. Then the effect faded as the courses went on. Scaled up, the effects were an order of magnitude smaller than earlier, smaller studies had suggested. The team even tried machine learning to predict which students would benefit from which intervention. It did no better than giving everyone the same one.
Kizilcec put it simply: „Behavioral interventions are not a silver bullet.“
I don’t read this as „nudges are useless.“ I read it as a warning about order of operations. A reminder email can bring someone back to a lesson. It can’t make a confusing lesson clear, and it can’t give a course a rhythm it doesn’t have. Nudges work on top of a course’s structure. They can’t replace it.
Four Numbers I Would Track Instead
If one completion rate is misleading, don’t replace it with a better single number. Track a small set that tells you where to act.
1. Completion among people who started. Drop the ones who enrolled and never opened lesson one. That is a marketing or onboarding question, not a course-quality question, and mixing the two hides both.
2. Completion by commitment level. Keep free and paid learners apart. If you run cohorts and self-paced access, keep those apart too. Averaging them together gives you a number that describes nobody.
3. The lesson where people stop. Build a simple chart: how many learners reached each lesson. A gentle slope is normal. A cliff after one specific lesson is a gift, because it tells you exactly where to look. Usually that lesson is too long, holds two ideas instead of one, or asks for effort before the learner has seen any result.
4. What happens in the first week. Whether a learner does anything real in the first days, finishes a lesson, submits a small task, asks a question, is the earliest signal you get. It arrives while you can still do something about it, instead of at the end of the course when all you can do is write a post-mortem.
None of these needs expensive analytics. Lesson-level progress is usually visible in the LMS you already use. A spreadsheet handles the rest.
What Changes When You Measure This Way
The biggest change is that you stop rebuilding whole courses. A cliff at lesson three means you fix lesson three. A weak first week means you fix the start: a clear first step, a quick win, a reason to come back tomorrow.
Structure does most of the work here. A course built around one clear result, broken into modules that each close one sub-task, with lessons that hold one idea each and open on a rhythm, gets finished more often than a course where „everything is there.“ If you are rebuilding with that in mind, here is a step-by-step breakdown of how to structure a course around a result, from modules to lessons to knowledge checks.
The same logic works in corporate learning. Mandatory compliance training and voluntary upskilling have different denominators. Reporting one blended completion figure to leadership hides where the real problem is.
A Note On Honesty
There is a trap on the other side. Once you know that the denominator moves the number, it is tempting to pick the flattering one and stop there.
Don’t. Report all of them, and say which is which. The 30% and the 48% from the Open Praxis study are both true. They answer different questions. Your job is to know which question you are asking before you decide your course is broken.
What To Do This Week
- Pull enrollment and lesson-level progress for one course.
- Split learners who never started from those who did.
- Separate free from paid, and cohort from self-paced.
- Chart how many learners reached each lesson and find the steepest drop.
- Look at the first seven days of your last intake: who did something, and who didn’t.
Then fix one thing. Just one. Measure again with the next group.
References
[1] Celik, B., & Cagiltay, K. (2024). Uncovering MOOC Completion: A Comparative Study of Completion Rates from Different Perspectives. Open Praxis, 16(3), 445–456. https://openpraxis.org/articles/10.55982/openpraxis.16.3.606
[2] Reich, J., & Ruipérez-Valiente, J. A. (2019). The MOOC Pivot. Science. Summary: „Study offers data to show MOOCs didn’t achieve their goals,“ Inside Higher Ed, January 16, 2019. https://www.insidehighered.com/digital-learning/article/2019/01/16/study-offers-data-show-moocs-didnt-achieve-their-goals
[3] Kizilcec, R. F., Reich, J., Yeomans, M., et al. (2020). Scaling Up Behavioral Science Interventions in Online Education. Proceedings of the National Academy of Sciences. Summary: „Study: No single solution helps all students complete MOOCs,“ Cornell Chronicle, June 15, 2020. https://news.cornell.edu/stories/2020/06/study-no-single-solution-helps-all-students-complete-moocs