August 2007
Organizing Your Thinking About Evaluation
One way to think about evaluating the effectiveness of your safety training is to use a model that provides concrete definitions of what can be learned from them. Kirkpatrick's four-level framework is one example that can be used. This framework consists of four levels that progress in difficulty from 1 (the easiest to conduct) to 4 (the hardest). When choosing the levels to include in your assessment, start by identifying the questions your evaluation needs to address.
KIRKPATRICK'S FOUR-LEVEL EVALUATION SCHEME
Level Measurement focus Questions addressed
- - Reaction Trainees' perceptions What did trainees think of this training?
- - Learning Knowledge/skills gained Was there an increase in knowledge or skill level?
- - Behavior Worksite implementation Is new knowledge/skill being used on the job?
- - Results Impact on organization What effect did the training have on the organization?
Examples:
Level 1: One way to assess trainee reactions and attitudes is to use a questionnaire.
Questions can gather opinions about training methods, the instructor, the environment in which training took place, or other aspects of the training process.
Pencil-and-paper surveys are convenient to use for trainees and the evaluator.
Level 2:Written or performance tests can assess change in knowledge/skills.
The best way to measure changes in knowledge or skills is to test trainees before and after training.
Note that even if a positive change is found, it is possible the trainees gained the new knowledge or skill from a source other than the training.
If it is not possible to test trainees before training, their performance can be tested after training and they can be asked whether or not their understanding or skill came from the training session.
Level 3: Post-training testing or observations can assess use of skills on the job.
This level must be completed outside of the classroom after trainees have had an opportunity to use what they have learned.
This level is more difficult because it requires trainers or some other evaluator to follow-up months after training.
Level 4: Quantifiable measures are often used when assessing organizational impact.
Some examples of measures that can be used are numbers regarding sales, injuries, or productivity.
It can be difficult to determine the extent to which other factors besides training (i.e., economics of region) may have contributed to changes in organizational performance.
Key Points to Remember:
Different aspects of training can be evaluated.
Level 1: Trainees’ perceptions
Level 2: Knowledge/skills gained
Level 3: Worksite implementation
Level 4: Impact on organization
Remember to direct your evaluation to specific questions.
How did the trainees react to the training?
Do people seem to have increased their skills?
Are new skills being used?
Has the organization been impacted?
Reference: Kirkpatrick, D. 2001. The four-level evaluation process. Ch. 12 in What Smart Trainers Know: The Secrets of Success from the World’s Foremost Experts, L.L. Ukens, ed. San Fransisco: Jossey-Bass/Pfeiffer, pp. 122-132.
 Be an OSHAcademy Booster Don't forget to tell your friends about OSHAcademy and all the great resources.
Be sure to visit OSHAcademy Safety Training Network. We're always adding new information and training helps. Be sure to visit a sponsored links to help keep the training free!
© 2007 Geigle Communications, LLC. All rights reserved.
|
|
