Blog

Supercharge Your Training Evaluations

Photo Credit: John Fowler, published under CC.

What is your goal in the organization? Why do they hire you to train people? Ultimately, it is to further the goals of the organization. These goals can be very diverse, from making more profit (e.g. in a commercial enterprise) to better helping people in need (e.g. by providing humanitarian aid).

But how do you know your training really adds to the bottom line? You evaluate your training results against the aforementioned goals. With the rise of the world wide web and recently mobile internet, this has become a lot easier.

Not only do you now have fully automated testing tools to measure the effectiveness of training, but you can also use better tools for ‘transfer of training’. That is, you can test if your trainees are implementing the newly acquired skills and knowledge in the workplace. But before we get into that in more detail, let’s take a look at all the steps involved in supercharging your training evaluations.

1. First of all, get to know the goals of the organization. If management, or the customer, is hesitating to share this information with you, ask them: How can I help further your goals by designing an effective training if I don’t know what you’re aiming for? Now is also the time for the organization’s line managers to establish the baseline to measure against: how is the organization currently doing, with regard to the goals?
2. Translate the organization’s goals into employee competencies. What are the employees expected to know or do and what skills do they need for that? Consult the employees themselves as well. Ask the employees “What do you need to learn to do an even better job?” Use an automated questionnaire tool to gather the information, but elicit responses through open questions.
3. Measure the gap between the required competencies and the current knowledge and skill levels. This entails developing measurements first, such as assessments, quizzes, and 360 degree feedback.
4. Once you have established the norm and the current deviation of the norm (so to speak), you’re ready to design your training. Your training should strive to close the gap between the expected competencies and the current knowledge and skill levels. Keep in mind that, as a group, trainees may have varying degrees of competencies. So ideally your training should be adaptive in nature, catering to the individual needs of the trainee. Online training is perfect for this but may not be applicable for more practical, hands-on kinds of training.
5. Perform the actual training. Start by referring back to the input the employees gave you when you asked them what they needed to learn. At the end of the training, use an automated questionnaire to evaluate the training.
6. Evaluate learning and behavior. Did your trainees actually acquire new skills and knowledge? Are they employing their newly acquired competencies in the workplace?
7. Evaluate the accomplishment of the organization’s goals. In step 1, you established the baseline. Some time after the training, ask the organization’s line managers if there’s a noticeable improvement. Is there a positive ROI for this training specifically? (ROI stands for Return on Investment).

Supercharging Evaluation Levels

Photo Credit: ed ouimette, published under CC.

Recognized Kirkpatrick’s 4 levels of evaluation in the list above? If you’re not familiar with his work on this topic: it’s a taxonomy of criteria (levels) for evaluating training. Here’s a quick overview of each level of evaluation:

1. Reaction: how is the training received? Use a questionnaire to find out.
2. Learning: what did the trainees learn? A thorough online test (‘quiz’) should answer this question.
3. Behavior: are they employing their newly acquired skills and knowledge on the job?
4. Results: does the organization meet its goals thanks to the training?

Levels 1 and 2

Levels 1 and 2 (reaction, learning) are traditionally the easiest evaluation levels. You can use paper and pencil or online tools to gather the data. A learning management system (lms) has the advantage of integrating the data with other options. For instance, if an online test (level 2 evaluation) shows a trainee’s skills  to be lacking, the lms may immediately offer additional online training to remediate the problem. Of course, if all of your trainees flunk the test, you should take a hard look at your training design first.

Level 3 Evaluation

Traditionally, level 3 evaluation (how is the training applied on the job?) takes place by simply having the line manager ask the employees: “You know that training you went to a few months ago? Are you applying it in your job and if so, give me an example.” Or, more formally, by doing assessments.

Here are a few additional ideas, made possible by the web and made easy by mobile devices:

• Automated 360 degree feedback: send out notifications to other employees to fill out the online feedback forms for their coworker. The same goes for any other type of assessment: put it online, this makes it easier to process the data.
• Online interactive training: put the theory behind the training online (this may not apply to practical hands on training). Have the trainees prepare the face-to-face sessions by inviting them online and pointing them to the documentation and interactive training. After the training keep recording usage of the online interactive training. This gives you some indication as to whether trainees are trying to actually implement the training on the job.
• Online work related tools: provide tools to effectuate the desired change; measure usage of said tools. Take sales presentation as an example. To prepare trainees for a sales presentation, the trainer has created a checklist. As before, put this checklist online. Make it a readily available tool and keep track of who’s using it and how frequently.
• Peer training facilitation: provide tools to disseminate information among coworkers: are your trainees spreading the word once they’re back on the job? Keep records to measure long term training success.

Level 4 Evaluation

Photo Credit: Ken, published under CC.

Level 4 is about evaluating the accomplishment of the organization’s goals. This is actually a job for line managers (or management in general), but part of it should be of interest to you as a trainer, too. Namely, is there a positive return on investment for the training specifically?

In addition, you should probably be aware of competency management software. This is a tool to keep track of employees’ competencies. Some learning management systems (lms) offer this option as well. It’s probably easier to correlate the success of an organization with the knowledge and skills of the employees with competency management software.