Two questions any eLearning designer should ask when designing a course are, “How will I know if this eLearning course is effective and how will I define effective?” Certainly every professional, no matter what the field, is concerned about doing great work and knowing how he/she will know when that great work is achieved. A surgeon wants to know whether a procedure is effective. A mechanic wants to know whether a repair works. An eLearning designer wants to know whether a training course is effective.
In some fields the question is easy to answer and in others it is more difficult. If a surgical procedure is effective, the patient recovers. If a car repair is effective, the car operates properly.
Effective eLearning is not always so clear. In other words, how do one really know whether that new product eLearning course was effective? The good news is that there is a model that designers can use to measure the effectiveness of training.
According to Jack Phillips, there are five ways to determine whether training is effective:
Level 1: Were learners satisfied with what they learned? Customer satisfaction matters.
Level 2: Did participants learn what the course was about? Did they pass the test at the end?
Level 3: Did behavior change? Are participants using the new technique or skill that was taught in your course?
Level 4: Did some element of performance improve? Did sales go up? Did the number of HR complaints fall?
Level 5: Was there a positive return on the investment made in the training? Was there a financial benefit that outweighed the cost of the training?
Presumably, if you were to go through the process of evaluating your eLearning course, using the steps above, you could make a good judgement about how effective your training was. Moreover, if you followed your instructional design process well, you built in time to do this evaluation from beginning to end. In fact, estimates suggest that the training evaluation process should be about five percent of your total budget (time and resources) for a particular course.
The problem is that few designers actually perform a complete training evaluation. Statistics from the ASTD State of the Industry Report show that the majority of companies surveyed perform a Level 1 evaluation, and that almost none of the companies surveyed perform a Level 3, 4, or 5 evaluation.
This is not surprising because it takes a significant effort to conduct an evaluation of levels three through five. Furthermore, Even if you do conduct levels four and five, it is very easy to challenge assumptions that link a training course directly to business results and financial gains.
Measure What Your Company Values
My advice is to evaluate your eLearning course based on what the culture of your organization values. If you work for an accounting firm and numbers rule, I would focus my evaluation on the return on investment of a training course. If you work for a company that values customer satisfaction, I would focus on Level 1.
There are many ways to evaluate whether your eLearning is effective. In fact, I heard once from a fellow learning designer, “If you want to know whether your training is effective, make it voluntary.”
Well, that was easy.
How do you know if your e-Learning courses are effective?
Bill Cushard, author, blogger, and learning experience (LX) designer, is a human performance technologist (HPT) with extensive, in-the-trenches experience building learning organizations in start-up and hyper-growth organizations like E*TRADE, the Knowland Group, and Accenture. You can follow him on Twitter or on Google+.