How to Measure ELearning Effectiveness: Using key performance indicators to validate investment decisions
For any business endeavor to be justified, business leaders ask their management teams to provide concrete facts and figures, in terms of Return On Investment (ROI), that show beyond doubt that the proposed initiative is worth undertaking. Similarly, once the project has been commissioned, senior management and project sponsors need to know whether their investment was justified – in hindsight. Therefore, to rationalize eLearning projects prior to development, Instructional Designers and project managers need to collect, evaluate, and report on specific Key Performance Indicators (KPIs).
How Key Performance Indicators Help In Justifying Αn eLearning Investment
If an eLearning initiative offers a better return than the outlay (dollars, time, effort, inconvenience factor) needed, then it is likely to be approved. However, when judging an eLearning projects’ ROI, business leaders expect to see tangible performance results following the implementation of the training program. These metrics are called Key Performance Indicators (KPIs) and are considered a crucial component of any ROI calculation.
Quite often though, trainers and Instructional Design teams focus on indicators that don’t rank too highly on an investment decision. For example:
- Number of courses developed.
- Number of hours of training delivered.
- Number of employees enrolled.
- Percentage of trainees that have successfully completed the entire training course.
While all of the above are extremely critical KPIs to monitor and track from a training perspective; when it comes to ROI in eLearning initiatives, they often take a back seat. In order to be able to justify eLearning investment, trainers need to use KPIs that directly link training to quantifiable performance indicators in the workplace.
Using Meaningful Key Performance Indicators
The importance and relevance of KPIs will vary from industry to industry, and even from one company in the same industry to another. That’s why, as part of the KPI identification process, it is extremely important for instructional designers to meet, and openly discuss the issue with senior project sponsors.
For instance, in the financial industry, where transactions are often executed in the millions, overly focusing on training that ends up reducing “rounding errors” to the nearest cent isn’t a very effective KPI. The same KPI (training with a focus on reducing errors), however, would be extremely meaningful in a hospital or health care setting, where even a single error could result in drastic consequences.
Here are 12 Key Performance Indicators, grouped into 3 performance-related segments, which trainers should measure as part of the ROI process for an eLearning project:
Performance Bottom Line Indicators.
In a sales-oriented organization, for example, the bottom line in judging performance is sales. This could either be:
- Number of sales orders secured.
- Dollar value of sales made.
- The number of new prospective clients enlisted.
Therefore, understanding the target audience for the training and the expected measurable performance results that training should deliver is essential.
Performance Improvement Indicators.
Another set of KPIs that instructional content developers must track relates to performance improvements that an eLearning course will result in. In the case of the same sales-oriented company above, such KPIs might be:
- Time spent prospecting a client.
- Number of errors/omissions during the sales process.
- Effectively identifying prospects’ needs (product selection).
- Time spent revising quotes (due to staff induced errors).
When designing eLearning content that measures such KPIs, the goal must be to show that the type of material being taught will result in performance improvements that can be measured.
Performance Quality Indicators.
In some industries, these are the hardest to quantify, yet performance quality KPIs are often a decisive set of measurements that could swing a project’s ROI in favor of the trainer. That’s because every organization seeks quality improvements within its rank and file, and any eLearning training that helps achieve that goal will be looked upon as a great investment.
Improving quality may come as a result of:
- Fewer customer complaints.
- Less scrap material produced during the production process.
- Reduction in the percentage of returned goods.
- Declining customer order cancellations.
- Increase in customer compliments/referrals.
When addressing such KPIs however, eLearning content must be very focused in its scope. For instance, in order to reduce/eliminate negative customer feedback, you need to demonstrate that the training program will include specific content to fill gaps that currently result in customer complaints.
Creating Effective Key Performance Indicators
In the previous segment of this article, we talked about the importance of coming up with meaningful KPIs that can aid in deciding whether an eLearning project is worth the investment. There is a defined methodology that eLearning content developers can follow. However, here are some highlights of this process:
Understanding KPI-Related Gaps.
Training content developers must start by understanding what gaps the organization needs to fill through training. This will give them the insight needed to define appropriate content to fill those gaps.
Measuring Current Performance Gaps.
You then need to think about metrics for measuring these gaps. For instance, if customer complaints seem to be the challenge, then instructional designers must measure the number of complaints, type of complaints as well as the most frequent reasons customers complain. This will help further refine your content roadmap.
Quantify And Define The KPIs.
Using the first two inputs, you must now define what the KPI will be and put measurable metrics in place. For instance, instead of saying “Deal with customer complaints issues,” instructional designers should define the metrics of what the proposed eLearning content will achieve as clearly and specifically as possible. For example, “Reduce customer order-taking related complaints by 70%.”
Build Content Around The KPIs.
You can now start to develop content, quizzes, simulation exercises and tests that specifically address each of the KPI statements.
Measure And Report On Post-Training KPIs.
Finally, you must include post-training monitoring and measurement, of specific functions or processes that the training addressed, to see if the KPIs were met.
Making KPIs Work For You
If the eLearning content you’ve developed helps address all (or most) of the KPIs you had set out to deal with, then management will consider it to be a great return on their investment. The most important take away for Instructional Designers is to be realistic in what the training content can achieve, and ensure that both, the issue to be addressed as well as the improvements being promised through training can be easily quantified.